Inspiration
Medical reports are written for doctors, not patients. When a scared person receives an abnormal result, the first thing they do is Google it — and spiral. We believed there was a better way: what if your health data could be transformed into a warm, human story that you actually understood? That question became Vitale.
What It Does
Vitale is a multimodal creative storytelling agent that transforms medical report PDFs into personalized illustrated narratives. Upload your report, choose your theme — Child or Adult — and Vitale's creative director AI generates a streaming story with typewriter narration, Imagen 3 illustrations appearing inline, synchronized voiceover, and a final Ken Burns story film. It ends with a personalized doctor questions card tailored to your actual findings — so you walk into your appointment prepared, not panicked.
How We Built It
We built Vitale on a FastAPI backend deployed to Google Cloud Run, using Gemini 3.1 Pro's native interleaved output as the core storytelling engine. A single Gemini prompt streams alternating [NARRATE] and [ILLUSTRATE] blocks in real time. Each narration chunk is passed to Google Cloud TTS for voiceover synthesis. Each illustration prompt is passed to Imagen 3 for AI image generation. The frontend — built in pure vanilla HTML, CSS, and JavaScript with no frameworks — receives everything via Server-Sent Events and renders each story page live as it arrives. A Canvas API and MediaRecorder pipeline then assembles all illustrations and audio into a downloadable Ken Burns story film entirely in the browser.
Challenges We Ran Into
The hardest challenge was reliable parsing of Gemini's interleaved stream in real time — buffering incomplete chunks without blocking the SSE stream required careful state management. Synchronizing audio duration with Ken Burns animation timing on the canvas was equally tricky — we derived display duration directly from TTS audio byte length to drive frame timing without any external libraries. Getting Imagen 3 and Cloud TTS to work seamlessly within Cloud Run's environment, including service account permissions and Secret Manager integration, added significant complexity for a tight build window.
Accomplishments That We're Proud Of
We're proud that Gemini's interleaved output isn't just a checkbox — it IS the product. Every word, every illustration, every story beat flows from a single streaming Gemini response. We're also proud of the frontend experience: a full particle background with mouse repel physics, Ken Burns animations, typewriter narration, and a cinematic story film — all in a single self-contained HTML file with zero dependencies. Most of all we're proud that Vitale treats a medical report not as data to display, but as a story worth telling.
What We Learned
We learned that Gemini's interleaved output is genuinely powerful when you design around it rather than on top of it. We also learned that the most impactful AI applications aren't the most technically complex — they're the ones that solve a deeply human problem with the right layer of intelligence. Structuring the creative director prompt to produce consistent, parseable interleaved output required far more iteration than any infrastructure challenge we faced.
What's Next for Vitale
We want to expand Vitale to support more report types — radiology, cardiology, genomics — each with its own creative theme and illustration style. We're exploring a Senior theme with larger text, slower narration, and warmer visuals for elderly patients navigating complex diagnoses. Longer term, we see Vitale becoming a bridge between clinical data and patient understanding at scale — partnering with hospitals and diagnostic labs to make health literacy a standard part of every report delivery.
Log in or sign up for Devpost to join the conversation.