CrisisDoc

• 

  Landing Page

Inspiration

Incident reports take 3-4 hours to write and are often incomplete. Workplace injuries alone cost businesses $170 billion/year in the US. We asked: what if you could just describe what happened and get a complete multimedia documentary back?

## What it does

CrisisDoc takes a simple incident description (or a live camera investigation) and produces a narrated multimedia documentary with:

• Text — Engaging, storytelling-style narrative (not a dry report)
• Images — AI-generated scene reconstruction diagrams, damage assessments, and visual timelines
• Audio — Documentary-style TTS voiceover for every section, with a "Play Story" button
• Video — A Veo-generated CCTV-style incident reconstruction

All four modalities are woven together in one interleaved output stream.

It also features a Live Investigation Mode — point your camera at a scene, talk through what happened, and Gemini Live responds in real-time with voice, asking follow-up questions like a senior investigator.

## How we built it

4 Gemini models, 1 pipeline:

| Model | Role | |-------|------| | gemini-2.5-flash-native-audio | Live bidirectional video + audio investigation | | gemini-2.5-flash-image | Interleaved text + image report via response_modalities=['TEXT', 'IMAGE'] | | gemini-2.5-flash-preview-tts | Documentary narration voiceover | | veo-3.1-fast-generate-preview | Incident reconstruction video |

Stack: FastAPI + WebSocket backend, vanilla JS frontend with WebRTC for camera/mic, Google GenAI SDK, deployed on Cloud Run with Terraform IaC.

TTS narration and video reconstruction run in parallel via asyncio.gather() to minimize wait time.

## Challenges we ran into

• The Gemini image generation model name changed — gemini-2.0-flash-preview-image-generation no longer exists. We had to query client.models.list() to discover gemini-2.5-flash-image supports interleaved output.
• The Live API only works with bidiGenerateContent-capable models — only the native-audio variants support it.
• Veo video generation is async (returns an operation to poll), unlike the synchronous content generation APIs.
• Browser audio handling required an AudioWorklet for PCM capture and manual WAV blob construction for playback.
• The Part.from_text() API requires keyword arguments (text=...), not positional — cost us a debugging session.

## What we learned

• Gemini's interleaved output (response_modalities=['TEXT', 'IMAGE']) is genuinely powerful — the AI decides where to place diagrams within the narrative
• Combining multiple Gemini models in a pipeline (Live → Image → TTS → Veo) creates experiences no single model can deliver
• The Google GenAI SDK is the fastest path for hackathons — ADK adds complexity without solving the core problem

## What's next

• Real-time streaming of the report (currently waits for full generation)
• Multi-language support for international incident reporting
• Integration with insurance claim systems
• Mobile-first PWA for field investigators

Built With

• cloud-storage
• css
• fastapi
• firestore
• gemini-2.5-flash
• gemini-live-api
• google-cloud-run
• google-genai-sdk
• html
• javascript
• python
• terraform
• veo-3.1
• webrtc
• websocket