-
-
AI pipline
-
SiteEye
-
tech stack will be used
Inspiration
What it does
Inspiration
Construction sites kill 48,000 workers in India every year. Safety rules live in documents nobody can open with their hands full. There is zero real-time AI support on most job sites today.
What it does
SiteEye is a real-time AI safety assistant running on Meta Ray-Ban Display smart glasses. It continuously monitors the site through the officer's camera, listens to their voice, and responds instantly through both audio and a heads-up display when a hazard is detected.
How we built it
Multi-model AI pipeline: frame quality checker using Laplacian variance algorithm, Sharp.js for image enhancement, GPT-4o Vision for scene understanding, RAG system using LangChain.js and ChromaDB for site-specific safety knowledge, Web Speech API for voice input and output, Bun WebSockets for real-time supervisor alerts, and SQLite for incident logging.
Key features
- 3-trigger hybrid capture: timer + voice + motion spike detector
- RAG-powered site-specific safety rules and equipment knowledge
- Dynamic capture frequency — AI increases monitoring when risk is detected
- Multilingual support for Indian construction workers
- Real-time supervisor alerts with frame snapshots
- Full incident audit trail
Track
Track 1 — Physical AI
How we built it
Challenges we ran into
Accomplishments that we're proud of
What we learned
What's next for SiteEye
Built With
- bun
- chromadb
- gpt-4o-vision
- langchain.js
- openai
- react
- sharp.js
- sqlite
- typescript
- vite
- web-speech-api
- websockets
