Inspiration
We got our inspiration from the frustration of seeing an object and not knowing where to find it or what it is. We wanted to create a tool that bridges the gap between the physical and digital world.
What it does
This system detects objects through the camera and immediately pull up useful information about them using APIs. It transforms what you see into searchable, enriched data. It is like having a smart visual search engine in real time.
How we built it
We used YOLO (You Only Look Once) models for real-time detection and Gemini API to instantly fetch data about the object. We used Roboflow to train and refine our custom model using active learning to improve detection accuracy for our specific target objects. We also integrated the Google Custom Search API to fetch purchase links for our objects.
Challenges we ran into
We had major issues with the API key's quota being exhausted too quickly despite having a paid plan. We also ran into Python environment issues, specifically incompatibilities with Python 3.14 and broken venv setups which cost us significant setup time.
Accomplishments that we're proud of
We're proud of successfully building an end-to-end pipeline that goes from a live camera feed to real-time object detection to enhance API responses in a seamless flow. We're also proud of training a custom model on our own dataset using Roboflow and integrating it with multiple APIs under time pressure.
What we learned
We learned a lot about managing Python environments, API rate limits, and the challenges of real-time computer vision. We also gained hands-on experience with YOLO model training, active learning with Roboflow, and chaining multiple APIs together to create a cohesive product.
What's next for WeHack 2026
We want to expand the model to detect a wider range of products and possibly be able to detect an object that is within the screen (e.g., detecting an object shown on a smartphone). We would also like to build a mobile app version so users can point their phone camera at any objects and get instant results.
Log in or sign up for Devpost to join the conversation.