-
-
Sentinal-Ai asssitant
Inspiration
Emergencies are chaotic, and in the first critical minutes people often panic, hesitate, or make unsafe decisions because they lack clear guidance. We were inspired by the gap between the moment an emergency occurs and the arrival of professional help. Sentinel-AI was created to explore how multimodal AI can responsibly support people during that gap by providing calm, structured, and accessible guidance using images, voice, and natural language, while always prioritizing safety and escalation to real emergency services.
What it does
Sentinel-AI is a multimodal emergency response co-pilot that helps users understand and respond to emergency situations. Users can upload an image of the scene and describe the situation using text or voice. The system analyzes the visual and contextual information, assesses risk level, and provides clear step-by-step guidance. It includes hands-free voice interaction, a checklist-based protocol flow to reduce cognitive load, and a responsible emergency escalation feature that generates a concise incident summary and guides users to contact professional services when needed.
How we built it
We built Sentinel-AI as a web application using a React frontend and a Node.js backend integrated with Gemini’s multimodal capabilities. The frontend focuses on clarity and low-stress interaction, while the backend handles image and voice input processing, safety-aware prompt engineering, and response generation. We used browser-native APIs for voice recognition and text-to-speech, and designed the system so all emergency guidance follows strict safety constraints and confirmation steps.
Challenges we ran into
One of the main challenges was designing interactions for high-stress situations without overwhelming the user. We had to balance providing enough guidance while avoiding unsafe or overly complex instructions. Another challenge was integrating voice input and output in a way that feels natural but still allows user confirmation to prevent misinterpretation. Ensuring ethical emergency escalation without claiming to replace professional services was also a key design challenge.
Accomplishments that we're proud of
We are proud of building a responsible emergency-focused AI experience that goes beyond a simple chatbot. The integration of multimodal analysis, voice interaction, checklist-based guidance, and a safe emergency handoff flow demonstrates how AI can be used thoughtfully in safety-critical contexts. We are also proud of the overall UX, which is designed to remain calm, clear, and accessible even under pressure.
What we learned
Through this project, we learned that building AI for real-world emergencies is as much a design and ethics challenge as a technical one. Small UX decisions, such as wording, confirmation steps, and visual hierarchy, have a huge impact on trust and usability. We also learned how important it is to design AI systems that know their limits and escalate appropriately rather than trying to do too much.
What's next for Sentinal-Ai
Next, we plan to expand Sentinel-AI to mobile platforms, especially Android, to enable real phone-based emergency calls, location sharing, and offline fallback guidance. We also aim to add training and simulation modes for education and preparedness, as well as explore partnerships with institutions and organizations to deploy Sentinel-AI ethically at scale.
Log in or sign up for Devpost to join the conversation.