Inspiration

Our inspiration behind this project was to use every sponsor prize and API while still building an app that works. Instead of focusing on a few APIs and giving our app a limited amount of features, we experimented how each API and connector can be used to assist the other. For example, the Presage SDKs takes data from the phone's camera and we use that data to calm the user using music, speech, and puzzles using Gemini and Eleven Labs.

What it does

SmartCam is a wellness app that record your vitals from a selfie video and helps you act on them. The user experience begins with the home dashboard which displays all the vitals that the phone has measured. By begin the scan, Presage SmartSpectra takes a 35-second face scan that measures pulse, breathing rate, and facial expressions. Based on your stress level, the user is presented with a game accompanied with music. We wanted to maximize the ways we can help the user to lower their stress-levels, and as a result we have used a combination of music, games, and haptic feedback. You can take a photo of a mean and Gemini vision calculated the macros of the meals to ensure the user takes the appropriate food. Furthermore, Gemini makes a food plan and gives feedback on your daily intake. Finally, ElevenLabs allows you to choose from 16 different accents to give you fun facts, meditations, guidance, and more. Finally, you can use Solana Pay to buy things that you want online.

How we built it

Used native Kotlin and designed a liquid-glass UI with widgets that expand when tapped, swipe navigation, haptics, and more. Presage SmartSpectra SDK monitor vitals. Google Gemini handles chat, food photos, plans, music, and shopping. The voice chats use ElevenLabs, and the phone's built-in voice takes over if ElevenLabs can't be reached. ML Kit for barcodes, Android UsageStats, Device Admin, the step sensor, AlarmManager, and six ambient tracks composed for the app. Tiger Data (TimescaleDB/Postgres). Vitals are stored in a hypertable as time-series data, and accounts, state and food logs live in their own tables. Data is saved on the phone and synced in the background.