Preventing Sudden Infant Death Syndrome Service using Gemini

logo

Inspiration

Create a microservice that can easily be integrated into baby monitors to prevent Sudden Infant Death Syndrome.

What it does

Sysem receives a video and a set of predefined custom rules and outputs the information about which rules were broken, in which time intervals and the reason why they weren't fullfiled

How we built it

We made a microservice that receives a video and a set of predefined custom rules and sends them to Gemini 1.5 Pro model via API (which is available on Vertex AI). The Gemini then returns the results in a form of .json file with information about every rule: Is it OK or Broken. If the rule is broken, the Gemini will output the timestamps (in MM:SS format) when the rule was broken with the description why. For the purpose of the Hackathon, we also made a simple Web App so users can easily upload a desired video and write their own rules in order to interact with Gemini and see his answers about given rules on a specific video.

Challenges we ran into

Make a good universal prompt that would work the best with our use case and also with any other use cases that users would want to use this app for. Figure out the best and simplest way to integrate Gemini via API into our solution.

Accomplishments that we're proud of

Our microservice that checks safety of baby's sleeping environment can prevent sudden infant death We made a microservice so that the solution could be easily integrated into other applications/systems Our approach is open to other use cases - it can detect a number of other rules and problems, not only on video, but also on image and audio.

What we learned

How to work with Gemini, Gemini's API, and what Gemini can do. How to use Gemini to solve a real life problem.

What's next for Preventing Sudden Infant Death Syndrome Service using Gemini

Experiment with different prompts, test other Gemini models once they are released, add features like speech to text, integrate other models and solutions.