Inspiration
We drew inspiration from Placer.ai, Safegraph, and EarthCam - leaders in location analytics and real-time camera data. Their platforms illustrated how powerful live insights can be when paired with user-friendly dashboards and robust data processing pipelines. We wanted to bring that same dynamic experience into a project that would help users monitor campaigns and foot traffic in real time.
What it does
urbanlens is a real-time analytics platform that facilitates campaign and site monitoring using either newly installed cameras or publicly accessible live feeds. It continuously ingests video streams over HLS, detects objects (using YOLO), and then displays updated insights on a map-based dashboard. For an added layer of intelligence, we integrated LLM capabilities to let users explore the data in a conversational way.
How we built it
- Ingestion & Frame Extraction: We used Streamlink to pull frames from HLS streams. These camera URLs are configured in our location table, which can be updated through our UI.
- YOLO Object Detection: We have a pool of Python workers that each run YOLO for object detection, letting us process multiple camera feeds in parallel. To coordinate these workers and pass around tasks, we used Celery, which gives us a distributed architecture for quick and efficient processing.
- Data Pipeline & Storage: After detecting objects and relevant metadata, we push the results into a time-series table, capturing timestamps, locations, and object types. This makes historical analysis and real-time queries straightforward.
- Frontend: We built the frontend in React with Tailwind CSS. The map displays real-time updates, including the latest detections and insights drawn from the time-series data.
- LLM Integration: We integrated a language model to provide a more intuitive data exploration experience, enabling users to query the system about trends, counts, or camera-specific insights.
Challenges we ran into
- Minimizing Latency: Achieving near real-time insights meant optimizing our pipeline—especially the steps between frame extraction and inference—to keep data fresh on the dashboard.
- Passing Images to the LLM: We wanted to let the LLM analyze or describe certain images directly. But managing images stored in S3 buckets—without manually downloading them—was a hurdle. We ended up exploring pre-signed URLs and secure access methods, balancing performance with security.
Accomplishments that we're proud of
- Real-Time Map Updates: We successfully combined YOLO detections with a map-based UI so that users see event-by-event changes as they happen.
- Scalable Inference System: By leveraging Celery, we can spin up multiple workers, ensuring the system stays responsive even if more camera streams are added.
- LLM Integration: Bringing a language model into the mix allowed us to offer an interactive question-and-answer approach to analytics, making data queries more natural for non-technical users.
What we learned
- Parallel Processing: Distributing object detection workloads taught us the ins and outs of scaling Python workers with Celery.
- Securing Media for AI: We discovered best practices for securely passing S3-based image data to different AI components, balancing performance with security.
- End-to-End Observability: Building a real-time pipeline made us appreciate the importance of monitoring each stage—from frame extraction and inference to database writes and frontend rendering—to maintain both reliability and speed.
urbanlens showcases how real-time computer vision, combined with large language models and a robust data pipeline, can bring location analytics and campaign monitoring to life.
What's next for 11 (Side Tables) Urban Lens
We aim to develop software that delivers meaningful value to businesses, driving measurable impact on their operations and bottom line.
Log in or sign up for Devpost to join the conversation.