SipSafe

HomePage1
Dashboard
Drink Logger
Detected Tampering
Receipt
Profile
Text Alerts
SipSafe Logo

Inspiration

We live in a college town where the line between a fun night out and a dangerous situation is thin. Most "drink trackers" are just tracking how many drinks you've had but dont account for the reality of nightlife, such as risk with drink tampering or the fact that once you're imparied, you can't easily read a phone screen. We wanted to build a wingman that uses AI to intervene when judgement fails.

What it does

Leveraging Gemini Multimodal Vision, SipSafe scans beverages for anomalies like sediment or cloudiness while instantly identifying drink types via photo analysis.

Integrated Voice AI provides real-time verbal confirmation of your drink's contents and delivers immediate warnings if potential tampering or safety risks are detected.

A dedicated SOS button alerts your inner circle instantly. The system also monitors your BAC levels, auto-notifying contacts at 0.15% for awareness and 0.30% for critical, life-saving intervention.

How we built it

Frontend: Built with React Native (Expo)

Design: The UI uses a grunge, gen-z aesthetic.

Backend: A Node.js/Express server hosted on DigitalOcean, utilizing MongoDB Atlas for user data and Firebase for secure authentication.

AI Integration: Gemini 1.5 Flash for image reasoning and spoofing detection.

ElevenLabs: ElevenLabs text‑to‑speech to convert app text into natural sounding audio then played the MP3 inside the Expo app so users hear spoken feedback.

Challenges we ran into

We faced complex Git merge conflicts early on from simultaneous pushes to the main branch. We resolved this by implementing a stricter branching strategy.

Aesthetic Implementation: Designing for a "Grunge Aesthetic" presented a unique UI/UX challenge. Since this high contrast gritty style is rare in mobile development, we had to build custom visual assets and styling logic from scratch without standard templates or reference points.

Full-Stack Integration: Synchronizing the frontend with an evolving backend required some planning. To maintain development momentum while backend endpoints were still in progress, we hard coded pages temporarily to ensure the frontend was logic ready for the final integration.

Accomplishments that we're proud of

We successfully implemented a global state monitor that can "interrupt" the user with an emergency modal regardless of which screen they are on.

Getting Gemini to accurately identify "abnormal cloudiness" in a liquid through a smartphone camera lens.

Creating an edgy aesthetic that feels like a nightlife app rather than a boring tracker tool.

What we learned

We learned that traditional UX principles fail when the user is impaired. Designing for a sober user is easy but designing for someone with blurred vision and reduced motor skills is a completely different challenge. Switching from a touch heavy interface to a "Voice-First" feedback loop via ElevenLabs was an important part of our project.

Before this hackathon, we thought of AI as individual tools. We learned how to build a Multimodal Pipeline, where the output of one model (Gemini’s visual reasoning) serves as the immediate input for another (ElevenLabs’ voice synthesis). Managing that data in real-time taught us how to keep track of complex AI workflows.

We learned the ethical complexity of building a safety app. When do you take control away from the user? We had to decide on the "Nuclear Option" the BAC trigger. We learned that for a safety app to be effective, it sometimes has to prioritize the user's well-being over their current preference, which meant engineering a countdown that automatically alerts friends unless the user can prove they are coherent.