Spare

Spare combines wearable camera feeds, live object detection, and interactive 3D building views to help first responders and authorities understand unfamiliar spaces and make faster, safer decisions.

Comment

Inspiration

During emergency evacuations or missions, every detail and second matters for success. We realized that first responders and military personnel don't have enough time to assess every part of their surroundings in an emergency. We wanted to explore how a live camera feed and a 3d render could provide teams with useful information before, during, and after an emergency response.

What it does

Spare streams a live feed from a camera worn by a responder and labels objects in real time. The responder taps a sensor to start and stop recordings, which are rebuilt as an interactive 3D model of the space. Operators can explore the model, run an AI agent to find the best route for evacuation or entry, and relay it to the team on site. Scenes can be saved and reused to prepare for similar emergencies.

How we built it

  • Wearable rig: a Logitech Brio on glasses and an Arduino touch sensor, run by a Python service (Flask, OpenCV) that streams video over Wi-Fi and records on a tap.
  • Dashboard: React, TypeScript, and Three.js, showing the live feed, RF-DETR detections, and an editable 3D sandbox.
  • Reconstruction: a FastAPI and PyTorch backend using Depth Anything V2, SegFormer, and OpenCV camera tracking, plus a Gemini and Blender mode that builds a furnished room model.
  • Routing: a route planner that treats unseen space as risky, with an LLM agent that turns plain-English instructions into missions.
  • Hosting: Vercel for the site, Docker and Fly.io for the API.

Challenges we ran into

  • Our camera ran at 5 fps until we found OpenCV was silently dropping the compressed video mode.
  • The browser, Python, and the Arduino IDE kept fighting over the same camera and serial port.
  • Our touch sensor's presses were invisible until the sketch learned each pin's resting level.
  • Building 3D from a single moving camera, with blank walls and no depth sensor.
  • Connecting hardware, software, AI agents, and 3D reconstruction into one working pipeline in 36 hours.

Accomplishments that we're proud of

  • A full loop from hardware to 3D: tap the sensor, record, and the clip loads into the dashboard by itself.
  • Real-time object detection on a head-mounted camera, running on a laptop.
  • A route planner that is honest about areas the team has not seen yet.

What we learned

  • How to drive Blender from Python through an MCP server, so an AI model can build 3D scenes on its own.
  • How to stream live video and sensor data over Wi-Fi between two laptops with low latency.
  • How to connect an Arduino to a web app, so a physical tap controls software in the browser.
  • To measure instead of assume. Our worst bugs were silent fallbacks, like the camera quietly dropping to a slower video mode.

What's next for Spare

  • Test the 3D render on more rooms and larger buildings.
  • Make the wearable smaller by replacing the backpack laptop.
  • Combine several responders' cameras into one shared map.

Built With

Share this project:

Updates

Submission history