Gemini Blackbox

A multimodal forensic reasoning system that reconstructs real-world events from incomplete image, audio, and text evidence, explicitly modeling uncertainty and alternate hypotheses.

Comment

Inspiration

Real-world incidents are rarely documented cleanly. Investigators often work with fragmented evidence — a single image after the event, a short audio clip, or an incomplete witness statement. Most AI systems today either summarize inputs or provide confident answers without acknowledging uncertainty. I wanted to explore whether Gemini could instead behave like a forensic investigator: reasoning cautiously, combining multimodal signals, and explicitly stating where conclusions may be uncertain or wrong.

This project was inspired by how accident reconstruction, journalism verification, and incident review rely on structured reasoning rather than single-source truth.

What it does

Gemini Blackbox is a multimodal forensic reasoning system that reconstructs plausible sequences of real-world events from incomplete evidence.

The application accepts:

  • An image showing the aftermath of an incident
  • An audio clip containing ambient sounds or reactions
  • A short textual witness note

Using Gemini’s multimodal reasoning capabilities, the system:

  • Infers a structured event timeline
  • Explains causal relationships
  • Proposes a primary hypothesis and multiple alternate hypotheses
  • Assigns confidence scores to each hypothesis
  • Maps every conclusion back to supporting evidence
  • Explicitly states uncertainty and performs a self-audit to identify weak assumptions and missing information

This is intentionally not a chatbot. The system performs a single analytical run and produces a structured forensic output.

How I built it

The project was built using Google AI Studio and the Gemini 3 API, focusing on prompt architecture and system orchestration rather than retrieval or fine-tuning.

Key design choices include:

  • Two-phase reasoning * Phase 1: Independent parsing of image, audio, and text evidence * Phase 2: Event reconstruction, causal inference, and hypothesis generation
  • Strict output structuring to prevent conversational responses
  • Confidence calibration for all conclusions
  • A self-critique phase where the system evaluates its own assumptions and identifies evidence gaps

The user interface was built directly in AI Studio’s Build tab to create a fully testable, public-facing prototype.

Challenges I ran into

  • Preventing the model from hallucinating missing facts
  • Forcing explicit uncertainty instead of confident guesses
  • Designing prompts that encourage self-critique without altering conclusions
  • Balancing detailed reasoning with readable, judge-friendly output

A major challenge was ensuring the system behaved like an analyst, not a narrator.

What I learned

This project deepened my understanding of:

  • Multimodal reasoning beyond object recognition
  • Prompt architectures for constrained, non-chat applications
  • Designing AI systems that acknowledge uncertainty
  • Using Gemini as a reasoning engine rather than a text generator

It reinforced that strong AI systems are defined as much by what they refuse to assume as by what they infer.

What’s next for Gemini Blackbox

  • Temporal video input for richer spatial–temporal reasoning
  • Iterative re-analysis as new evidence is added
  • Reliability scoring for different evidence sources
  • Investigator training and journalism verification workflows

Built With

Share this project:

Updates