Gemini Live: The Intelligent Multi-Agent Kitchen Sentinel

Gemini Kitchen Sentinel architecture

The Inspiration

Gemini Sentinel (or Desk-Chef) was inspired by the necessity of bringing a bridge between the physical safety of the kitchen and the digital health monitoring. The details of working in a high-traffic kitchen are likely to induce distraction, which leads to safety risks, such as gas leakage or using hot pans without gloves. I thought I could come up with a "Watchful Companion" that would be a proactive sous-chef, whereby recipes like No Oil, No Boil, such as my tomato and carrot salad are to be adhered to the letter and that I would ensure that the standard of safety has been thorough. How I Built It

The project is developed based on Raspberry Pi 5 and a complex Multi-Agent System architecture:

Hardware Integration: I added a Camera Module 3 as a vision and MQ2 Gas Sensor to the GPIO 4.

Specialized Agents:

    Gas Agent: Checks the leakages within every 2sec.

    Safety Agent: Uncovers the use of Gemini 2.5 Flash vision to identify the presence of ungloved pan handling.

    Ingredient Agent: Examines the recipe of some hidden oils or too much salt/chilli.

Gemini Live: I applied the Gemini Live API in order to build a voice-first, real-time interface in which the user is able to talk to the AI in a natural manner as they cook.

The one that does the Orchestrating: This agent is the brain, and he or she adopts the critical gas warnings, rather than trivial warnings concerning ingredients.

Challenges I Faced

Concurrency & State Management: To handle five agents at once a strong thread-safe Kitchen State with locks was needed to stop data being corrupted.

Real-Time Vision Latency: It was a challenge to balance high-resolution images capture to analyze the ingredients with the necessity to have low-latency safety alerts.

Environment Noise: The limiting of the Gemini Live voice-activity detector (VAD) to avoid detecting kitchen noise, but still hear the user voice commands, was a major obstacle.

What I Learned

I learned the key concepts of Agentic Workflows and the way to organize several AI professionals around one task. I likewise acquired how to optimize Generative AI on edge devices such as the Raspberry Pi, namely how to manage API limits and error recovery of long-running processes. Above all, I learned how AI can become more than merely a passive tool and make a proactive partner of safety.