Newton’s Lens

• 

  this is front page of my project in which

Inspiration

Newton’s Lens was inspired by the gap students face when performing science experiments alone without a teacher’s immediate guidance. In classrooms, teachers instantly spot incorrect wiring, wrong measurements, or unsafe setups, but in remote learning, students only realize mistakes after experiments fail. Seeing LEDs burn out, ramps misaligned, and overheated beakers made us ask a simple question: what if a camera could understand the experiment like a teacher and warn the student before anything goes wrong? That idea—turning a device camera into an intelligent scientific observer—became the foundation of Newton’s Lens.

What it does

Newton’s Lens acts as an AI lab partner. It observes a student’s experiment through a live camera feed, identifies lab components, understands how they are connected, predicts the physical outcome using scientific laws, and provides real-time guidance. For example, if a battery is connected directly to an LED without a resistor, the system predicts excessive current using Ohm’s Law:

[ I = \frac{V}{R} ]

and immediately warns the student before the LED burns. In mechanics experiments, it estimates ramp angles and validates motion using:

[ a = g \cdot \sin(\theta) ]

The student receives visual highlights on the screen and spoken instructions, creating the feeling of a teacher watching over their shoulder.

How we built it

We built Newton’s Lens with Python as the backend and React + TypeScript as the frontend. The camera feed is processed in Python using OpenCV, where custom computer vision techniques like color filtering, contour detection, edge detection, and shape recognition identify components such as batteries, LEDs, resistors, wires, ramps, balls, and beakers. The system then determines how these parts are connected using geometric proximity and stores these relationships in a graph using NetworkX. A rule-based reasoning engine applies scientific formulas to this graph to predict outcomes. The backend sends coordinates and guidance messages to the React frontend, which overlays highlights and instructions on the live video stream while also providing voice feedback through text-to-speech.

Challenges we ran into

One major challenge was moving beyond simple object detection to true scene understanding—teaching the system not just to see components but to understand how they interact. Detecting wire connections accurately through geometry was difficult due to varying camera angles and lighting. Maintaining real-time performance while processing video frames, analyzing connections, applying physics rules, and sending data to the frontend also required careful optimization. Integrating the Python backend with a responsive React interface without noticeable delay was another technical hurdle.

Accomplishments that we're proud of

We are proud that Newton’s Lens does more than recognize objects—it understands physical relationships and predicts outcomes before mistakes happen. The moment the system warns a student that an LED will burn out and highlights exactly where to add a resistor demonstrates the true power of the project. We also successfully combined computer vision, graph theory, and scientific reasoning into a smooth real-time experience with an interactive frontend.

What we learned

Through this project, we learned how powerful it is to combine perception (computer vision) with reasoning (scientific laws). We gained experience in graph modeling, real-time video processing, and full-stack integration between Python and React. Most importantly, we learned that AI becomes far more impactful when it understands why something happens, not just what it sees.

What’s next for Newton’s Lens

Next, we plan to expand support for more types of experiments, improve detection accuracy under different lighting conditions, and introduce more advanced reasoning for chemistry and electronics. We also aim to optimize performance for mobile devices and explore augmented reality features to make guidance even more immersive, bringing Newton’s Lens closer to becoming a universal AI lab assistant for students everywhere.

Built With

• gemini
• html5
• javascript
• python
• react
• typescript