Inspiration

The idea was inspired by the research by Siao et al., who proposed a COVID-19 robot for assisting medical staff with patient guidance. Alomari et al. also evaluated the "effectiveness of the AI chatbot ChatGPT in triaging patients, focusing on its accuracy, safety, efficiency, and impact on patient care".

What it does

This hospital monitoring robot uses Gemini 3 as its core intelligence engine, enabling continuous, real-time patient monitoring and predictive emergency detection. Gemini 3’s multimodal reasoning fuses live video, audio, and staff-provided clinical context, ensuring the system never guesses patient identity or condition. The system analyzes posture, immobility, agitation, and respiratory effort, along with audio cues such as distress sounds, to generate real-time, explainable alerts using the video monitoring feature. Gemini 3’s reasoning capabilities enable it to predict emergencies before they occur, including falls, sudden deterioration, or behavioral risks, giving staff critical lead time. Gemini 3 also supports bidirectional communication, keeping staff terminals synchronized with mirrored video, live reactions, and predictive alerts, while enabling staff to update context via text or voice. By integrating Gemini 3, the application achieves a cutting-edge, predictive, explainable, and fully synchronized patient monitoring system, demonstrating innovation, safety, and competition-grade readiness.

How we built it

We built the app using Google AI studio which provided us with the Gemini 3 API. We designed a detailed, specific prompt with all the features that we need for the app.

Challenges we ran into

The challenge we ran into was constantly running out of credits for the Gemini 3 API to run the AI vision monitoring system.

Accomplishments that we're proud of

We managed to fix a bug that was constantly occurring on the AI monitoring camera, whereby the link kept breaking. We iterated using advanced prompts to fix the issue.

What we learned

We learned that developing a solution is not a linear process, and you constantly need to iterate until you get the most optimal solution. Additionally, we have learned that Gemini 3 provides a robust and faster way to develop waterproof AI solutions. However, one still needs to pay attention to the code when developing a low/no code solutions.

What's next for Synctech Health

We will further test the app and develop it further by enhancing the back-end. The aim is for us to develop a final product that can be tested on a real system and monetized.

Built With

  • gemini-3
  • google-ai-studio
  • html
  • json
  • machine-vision
  • multimodal-inputs
  • next.js
  • prompting-engineering
  • text-based-prompting
  • typescript
  • voice-interaction-(conceptual-within-gemini-3)
Share this project:

Updates