Here’s a detailed continuation to complete the remaining sections of your GDG Hackathon project write-up for NOBI:
How We Built It
We built NOBI using a mix of hardware and cloud-based AI tools. For accuracy and speed, the object and face recognition system was developed with YOLOv8 and Google Cloud Vision API. For emotion and age detection, we used TensorFlow Lite models. Gemini Pro API manages natural language understanding and voice commands for interaction. The mobile companion app was created with Flutter, while Firebase handles user authentication, real-time database syncing, and cloud storage. We designed a modular system architecture to expand NOBI's abilities in cleaning, security, and organisation.
Challenges We Ran Into
- Integrating real-time video processing with low-latency feedback was challenging.
- Balancing on-device machine learning with cloud APIs to keep the system responsive and affordable.
- Training object detection models to tell apart commonly misplaced household items.
- Making UI/UX decisions to simplify the companion app while keeping full functionality.
Accomplishments That We’re Proud Of
- Achieved smooth integration of face, object, and emotion detection in one system.
- Created a modular Flutter app with real-time security and mood tracking.
- Built a friendly assistant that is both useful and kind.
- Enabled offline backups for key features like security alerts and object recall.
What We Learned
- The importance of user experience in AI-powered robotics: even strong features must feel easy to use.
- We can efficiently optimise real-time processing with YOLOv8 and Vision API.
- Gemini API is very effective for voice commands and chatbot functions.
- A well-organised Firebase backend can support a complex robot ecosystem.
What’s Next for NOBI
- Launch the MVP with select home users and gather real-world feedback.
- Add indoor mapping and navigation for autonomous room movement.
- Expand multi-user face profiles and mood journaling analytics.
- Integrate text-to-speech for conversational audio output.
- Explore commercial partnerships for smart home bundles or elder care integration.