Project J.
Inspiration
Our inspiration was from Formula 1 itself. The amount of strategy, engineering and gameplay that goes into every lap of the race amazes us and to create a system that contributes to the future of our favourite sport is the dream. But not just that, to go above and beyond, to make a system that can be used in an industry, and can be aligned for any scenario is the goal.
What it does
Project J is a domain-adaptive Visual Difference Engine that detects changes across time-series images visually and analyses, interprets and predicts its impact in real-time.
Detects constraint-based differences : For example, front and rear wings in F1 or aircraft wings and turbine blades pass static stress tests but deform at high speeds and aerodynamic pressure. Hence, we use multi-spectral imaging and optical flow tracking to detect real-time deformation under load and temperature constraints.
Detects manufacturing variations : In F1, even 0.5 mm deviation can give an advantage and in areas like chip fabrication and aerospace-assembly, sub-milli meter inconsistencies can lead to thermal hotspots and assembly misfits. To prevent this, we use 3D CAD analysis and semantic differencing.
Predicts temporal degradation : Over a race weekend, repeated heating and vibration can cause engine failure. Bridges, pipelines, wind turbines also suffer progressive-degradation until failure. Project J uses spectral imagery, temporal reasoning and LSTM-based forecasting for an Event Reasoning Engine to avoid this.
Because of these predictive detection models and real-time analysis, Manufacturing inspections, brand compliance monitoring, infrastructure assessment, on-track analysis processes are simplified, require much less manual labor and reduce part wastage and downtime.
How we built it
Phase 1 : We begin by setting up the core infrastructure using multi-camera arrays capable of capturing 4K footage at 100+ frames per second. Using OpenCV, we design an image processing pipeline that handles alignment, illumination normalisation, and initial difference detection in real time. This data is stored and managed through a PostgreSQL-based infrastructure, and a web-based dashboard allows us to visualise changes frame by frame. Within this phase, we achieve <1 second latency and build the groundwork for more complex, high-speed visual analysis.
Phase 2: Once the pipeline is built, we integrate advanced computer vision and machine learning engines. This is done by using SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features) for geometric feature extraction and optical flow algorithms to detect motion based deformations across frames. Using PyTorch and TensorFlow, we train deep learning models to classify detected changes by type and severity By this stage, we'll achieve 95% accuracy in defect detection and 90% precision in change classification.
Phase 3 : We introduce CAD model integration for physical-to-digital comparisons along with real-time alerting and visualisation dashboards for structural deformations and regulatory risks as and when they happen.
Phase 4 : Once detection is on-point, we build the predictive engines using Time-Series Analysis Engine that uses LSTM neural networks and ARIMA models to recognise degradation trends and forecast future failures.
Phase 5 : We finally package everything and deploy our project on both cloud and edge platforms with continuous monitoring and alert systems to detect macro-level deformations, reveal progressive faults and with seconds-based response systems in place for use in F1 and other industries.
Challenges we ran into
- Capturing 4k frames at high speeds requires us to introduce multiple cameras and use precise alignment and accurate visual differencing modules.
- Varying light intensities and reflections lead to false positives and outcomes. This requires us to introduce illumination normalisation algorithms to balance exposure without losing details.
- Small camera distortions cause mismatches while comparing with 3D CAD models. A hybrid feature-based 3D registration and physics based geometric models are to be installed.
Accomplishments that we're proud of
Creating a unified system that can monitor a car's front wing flex but also detect for a range of industries that are as large as manufacturing units and turbines to semiconductors. This achievement feels sweeter because most of these domain specifications can be met without rebuilding from scratch - just domain specific input and training. The upgrade from a simple capture to decision making, predictive module fusing RGB, thermal and telemetry data seems like a huge step forward. The reduction in energy usage, part wastage and the need for manual intervention is what we're most proud of.
What we learned
That perfect conditions don’t exist. To keep building, making mistakes, rectifying them and to keep repeating this loop is the only constant. The need for multiple inputs and constant cross-checking be it in project J or in real life. Incorporating time and sequence aware intelligence yields the most accurate results. To implement cross-domain fusion. TO make projects worthy of every sector, thus making technology and innovation much more sustainable. To never eliminate human oversight - it only amplifies decision making.
What's next for Untitled
Our next step focuses on taking our prototype to the global level - making an implementable intelligent inspection system that adapts across industries. Using industrial grade Raspberry Pi, NVIDIA Jetson modules and ESP-based microcontrollers to enable onsite inspection for factories, construction sites and race garages. To fuse LiDAR depth sensing, hyper-spectral imaging to fuse RGB and thermal data to detect structural anomalies like stress, heat etc. Then, we'll launch NOVA Data Commons, including synthetic data generated from CAD simulations and real-world examples to turn Project J into a real-world Predictive Decision Engine that not only detects time-series images but also suggests optimal execution plans, along with predictive maintenance suggestions. For Project J to be integrated in motorsport, aerospace, precision manufacturing, renewable energy and various other domains in what's next.
Built With
- arima
- cad
- esp
- jetson
- lidar
- lstm
- nova
- opencv
- postgresql
- python
- pytoch
- raspberry-pi
- sift
- surf
- tensorflow
