SightSync

Hardware Track

Inspiration

What inspired us to create SightSync was our interest in security technology, smart monitoring systems, and finding a way to make computer vision more accessible through a low-cost setup. We wanted to build something that could recognize objects in real time, estimate how far away they are, and eventually serve as the foundation for a tracking system that could be used in security, monitoring, or general object awareness applications.

We were also excited by the idea of combining both software and hardware into one project. We wanted to challenge ourselves by building something that felt practical, interactive, and expandable beyond the hackathon.

What it does

SightSync is a real-time object awareness system that uses computer vision to detect objects, track their movement, and estimate their distance from the camera. The system uses an iPhone camera feed as the video source and runs object detection and tracking through Ultralytics YOLO and OpenCV.

Right now, the project can identify objects in real time, assign tracking IDs to moving objects, follow movement across frames, estimate approximate distance for calibrated objects, and display all of that information in a live visual interface.

Our original goal was to take it a step further by connecting the detection system to a Raspberry Pi-powered pan-tilt mount so the camera setup could physically follow detected objects. Even though we were not able to fully complete that part within the hackathon timeframe, the current version still shows the computer vision foundation that would make that possible.

How we built it

We built SightSync by combining software and hardware ideas into one low-cost vision system.

On the software side, we used Python, OpenCV, Ultralytics YOLO, Camo to use an iPhone as a webcam feed, and GitHub for version control and backup.

We first focused on getting a reliable live camera feed into the project. After trying different options, we ended up using an iPhone connected through Camo, which gave us a flexible and better-quality video source. From there, we built a Python application that handled object detection and tracking in real time. We also added a simple distance estimation feature by calibrating known objects and using bounding box width as an approximate way to estimate distance.

On the hardware side, we explored integrating the system with a Raspberry Pi and a two-servo pan-tilt mount. The idea was to use the computer vision output to eventually control servo movement so the camera platform could physically follow the target object. Even though that hardware portion became more time-consuming than expected, it still shaped the direction of the project and what we want to build next.

Challenges we ran into

One of our biggest challenges was time. Since this project involved both computer vision and hardware integration, we had to constantly decide what was realistic to complete during the hackathon.

Another major challenge was getting the camera setup to work smoothly. We had to figure out how to use the iPhone camera as a live input source, and we ran into backend issues while trying to get OpenCV to properly read the Camo feed. We ended up testing multiple camera backends before finding a setup that worked reliably.

We also ran into hardware limitations while planning the Raspberry Pi and servo portion of the project. Powering high-torque servos safely and reliably required more planning than we originally expected, especially while trying to keep the setup low-cost and portable. On top of that, coordinating the software side with a custom physical mount added another layer of difficulty.

Accomplishments that we’re proud of

We’re proud that we built a working real-time computer vision system that can successfully detect objects, track them, and estimate distance using a relatively low-cost setup.

We’re also proud that we got a live iPhone camera feed working, integrated YOLO object detection and tracking, built a usable and demoable interface in a short amount of time, and created a strong software foundation for future hardware expansion. We also made sure to document and back up our progress through GitHub.

Even though we did not fully complete the Raspberry Pi pan-tilt implementation, we’re still proud that we pushed the project beyond a basic software-only idea and designed it with real hardware expansion in mind.

What we learned

This project taught us a lot about what it actually takes to build a computer vision system under time pressure. We learned that getting the model to run is only one part of the process. Making the camera input, tracking logic, calibration, and hardware plans all work together is what really makes a project challenging.

We also learned more about camera backends and live video troubleshooting, how object detection and tracking work differently in practice, how calibration can be used for approximate distance estimation, the importance of version control during fast prototyping, and how much extra planning goes into combining software and hardware into one system.

More than anything, we learned how important it is to build a strong MVP first and then expand from there.

What’s next for SightSync

The next step for SightSync is finishing the hardware integration. We want to connect the detection and tracking output to a Raspberry Pi-controlled pan-tilt mount so the system can physically follow detected objects in real time.

In the future, we’d like to add full servo-based object following, a more refined custom 3D-printed mount, improved distance estimation, and continue developing it into a more complete low-cost smart monitoring system that could be useful in security, robotics, or automation settings.

Built With

• python
• ultralytics
• yolo