Inspiration

🌍 About AI Reality Reconstructor

💡 Inspiration

Real-world investigations involve fragmented evidence scattered across CCTV footage, images, videos, documents, sensor logs, and other sources. Understanding what actually happened requires manually analyzing these sources, connecting relationships, reconstructing timelines, and identifying missing information.

We asked ourselves: What if AI could connect all these disconnected pieces and reconstruct a possible reality from available evidence?

This inspired us to build AI Reality Reconstructor — an AI-powered forensic investigation and 3D reconstruction platform that transforms fragmented evidence into a connected, interactive, and explainable representation of events.

Our vision is to move beyond traditional evidence management and create an intelligent system that helps investigators understand not just what evidence exists, but how different pieces of evidence connect and what they collectively reveal.

🚀 What It Does

AI Reality Reconstructor combines multimodal AI, temporal reasoning, relationship analysis, and spatial intelligence into one unified platform.

Key Features:

Multimodal Evidence Processing: Processes documents, images, videos, and sensor data using specialized processing modules, OCR, and computer vision.
Intelligent Timeline Reconstruction: Organizes extracted events chronologically and identifies potential missing timestamps, inconsistencies, and possible event sequences.
Evidence Relationship Graph: Connects people, objects, locations, and events to uncover relationships and correlations across different evidence sources.
AI-Powered Investigation Intelligence: Helps identify evidence gaps, missing information, and potential questions that could guide further investigation.
NVIDIA Nemotron-Powered Spatial Intelligence: Uses NVIDIA Cosmos-Nemotron-34B through Nebius Token Factory to interpret scene descriptions and object detections and generate structured spatial information.
Interactive 3D Reconstruction: Converts spatial information into interactive 3D environments where users can explore scenes, inspect objects, and understand spatial relationships.
Investigation Visualization: Provides timeline playback, object inspection, scene navigation, measurement tools, floor-plan visualization, and report export capabilities.

🛠️ How We Built It

We developed the project using a modular architecture combining AI-powered backend services with an interactive frontend.

Technology Stack:

Backend: Python, FastAPI
Frontend: React, TypeScript
3D Visualization: Three.js
AI Model: NVIDIA Cosmos-Nemotron-34B
AI Infrastructure: Nebius Token Factory
Computer Vision: OpenCV, YOLO, OCR
Data and Relationships: NetworkX, PostgreSQL, SQLAlchemy

System Architecture:

Evidence Processing Layer: Specialized modules process different evidence formats and extract relevant information.
Timeline and Correlation Engine: Extracted events are organized using temporal relationships and connected through an evidence relationship graph.
Investigation Intelligence Layer: Analyzes available information to highlight potential inconsistencies, missing evidence, and investigation opportunities.
Spatial Reconstruction Engine: Integrates NVIDIA Cosmos-Nemotron-34B through Nebius Token Factory using an OpenAI-compatible API. Custom prompts instruct the model to generate structured JSON containing room dimensions, object positions, bounding boxes, camera information, and spatial relationships.
Visualization Layer: Converts spatial information into Three.js-compatible scene data for interactive 3D reconstruction and 2D floor-plan visualization.

We used Nemotron out of the box with custom prompt engineering rather than fine-tuning. Its structured outputs allow downstream reconstruction components to consume AI-generated scene information.

📚 What We Learned

Building this project helped us understand how multiple AI capabilities can work together in a complex real-world application.

We gained practical experience in:

Multimodal evidence processing and computer vision.
Temporal reasoning and event correlation.
Large Language Model integration through hosted inference APIs.
Prompt engineering and structured JSON generation.
Spatial intelligence and 3D scene reconstruction.
Integrating independent AI modules into a unified architecture.
Building interactive 3D experiences using Three.js.

One of our most important learnings was that AI-generated interpretations must be distinguishable from verified evidence. We designed our approach around observed, inferred, and unknown information so that assumptions are not presented as established facts.

⚡ Challenges We Faced

1. Multimodal Evidence Integration

Different evidence sources have different formats, structures, and levels of reliability. Building a common architecture to process and correlate them was challenging.

2. Temporal Reasoning

Evidence may contain incomplete timestamps, conflicting information, or missing events. Building a meaningful timeline while accounting for uncertainty required careful reasoning.

3. Spatial and 3D Reconstruction

Translating natural-language scene descriptions and object detections into meaningful 3D coordinates, dimensions, and spatial relationships was one of our biggest technical challenges.

4. Reliable Structured AI Outputs

Our reconstruction pipeline depends on machine-readable outputs. Designing prompts that consistently generate valid JSON with the required spatial information required experimentation and validation.

5. End-to-End System Integration

Connecting independent processing modules, AI inference services, backend APIs, and interactive 3D visualization into one coherent application required a modular architecture.

🔮 What's Next?

We aim to improve spatial reconstruction accuracy, strengthen cross-source evidence correlation, introduce more advanced multimodal reasoning, and support richer real-world scene reconstruction.

Our ultimate goal is to make complex investigations more understandable by transforming fragmented evidence into connected, explainable, and interactive representations of reality.

AI Reality Reconstructor — From Evidence to Reality, Powered by AI.