Lantern

main page
asking for advice page
advice page
cv cam setup
yolo point localization demo
gpu machine

Inspiration

Our inspiration for Lantern came from observing the numerous security cameras scattered throughout San Francisco's Chinatown. We recognized these as a powerful, underutilized resource that could provide valuable insights to local businesses without compromising individual privacy.

What it does

Lantern taps into existing security camera feeds in San Francisco's Chinatown to create a real-time density heatmap of pedestrian traffic. Businesses can opt-in to access this anonymized, live data to enhance marketing strategies, optimize operational hours, manage staffing efficiently, or even implement dynamic pricing models based on foot traffic.

How we built it

We leveraged Flask to build the backend API server, combined with Python, PyTorch, and CUDA for GPU acceleration. YOLO (You Only Look Once) handles rapid, privacy-focused object detection—identifying human presence without storing faces or personally identifiable data. Positions detected by multiple cameras are triangulated using custom CUDA/PTX kernels, which rapidly compute precise global locations and filter duplicates, ensuring data accuracy and speed. Additionally, an Ollama-powered LLM generates dynamic business recommendations based on real-time data.

Challenges we ran into

Privacy:

Ensuring accurate analytics without compromising personal privacy was challenging. Real-time processing:
Efficiently managing and triangulating large data streams in real-time required careful optimization of CUDA kernels and GPU memory management.

Accomplishments that we're proud of

Creating effective, GPU-accelerated triangulation methods that run in real-time.
Implementing a privacy-preserving approach, with zero storage of personal or identifiable data.
Successfully integrating YOLO, CUDA, and Ollama into a cohesive real-time analytics system and pipeline that can run in real time!

What we learned

Advanced GPU programming with CUDA and PTX for real-time computations.
Effective methods for preserving individual privacy while still extracting meaningful analytical data.
How to build and optimize complex, multi-component real-time analytics applications.

What's next for Lantern

Moving forward, we plan to refine the system’s scalability, optimize our GPU kernels for even better performance, and expand Lantern’s capabilities to other neighborhoods. We also aim to enhance the precision of our analytics and further improve integration with business decision-making processes. We also intend to get access to a larger set of cameras to further test, improve, and refine our software.

Presentation Link: https://docs.google.com/presentation/d/1-nsrWjTf_XG5RbzGUwdzUKADb_v1uvwSLA0mWg6bjwY/edit?usp=sharing