Inspiration
Inspired by NVIDIA's breakthroughs in artificial intelligence, computer vision, and digital twins, we wanted to reinvent how the world approaches security. For decades, surveillance has been limited by fixed camera positions, disconnected footage, and blind spots. Investigators are forced to manually piece together what happened across multiple recordings. We asked ourselves: What if you could place a camera anywhere, even where no physical camera exists? What if AI could investigate an incident for you? That vision became Sentry4D.
What it does
Sentry4D transforms existing CCTV infrastructure into an AI-powered, interactive 3D security environment. Using LiDAR, computer vision, and synchronized camera footage, investigators can navigate a reconstructed room, rewind time, track movement, and investigate incidents spatially. Powered by NVIDIA Nemotron, Sentry4D also features an autonomous AI investigation agent. Instead of simply answering questions, Nemotron performs multi-step investigations, retrieves incident evidence through integrated tools, analyzes subject movements and camera observations, and generates structured investigation reports. Investigators can use the AI's findings to navigate directly to important moments in the 3D replay. Our prototype combines: • 3D Digital Twins: Explore a real LiDAR-scanned environment. • Four Synchronized Cameras: Investigate multiple feeds through one timeline. • Virtual Camera Navigation: Move freely through the reconstructed environment. • Spatial Person Tracking: Visualize movement in 3D. • X-Ray Mode: Examine tracked positions through obstructing geometry. • NVIDIA Nemotron AI Investigator: Automatically analyze incidents through multi-step tool-driven workflows. • AI-Generated Investigation Reports: Summarize events, supporting evidence, and recommended ## next steps. • Interactive Evidence Replay: Jump from investigation findings to relevant moments in the 3D scene. Our ultimate vision is to make physical camera placement irrelevant to how investigators explore an incident.
How we built it
We used an iPhone 16 Pro Max and Polycam to scan a real room, then positioned four cameras around the environment to record a staged incident. We built the interactive digital twin using React, TypeScript, Three.js, React Three Fiber, Vite, and Zustand. We developed camera-calibration tools to align physical recordings with virtual camera positions, synchronized all four videos to a master timeline, and mapped observed person movements into the reconstructed environment. We also integrated NVIDIA Nemotron through its API to power our autonomous investigation workflow. Nemotron uses tool calling to retrieve structured incident data, examine timelines, cross-reference camera observations, and generate evidence-backed summaries. The investigation interface connects these findings to the existing 3D replay, allowing users to move directly from AI analysis to visual verification. Claude Code accelerated development across our rendering, tracking, synchronization, and AI integration workflows. Challenges we ran into Camera calibration: Matching physical cameras to their virtual counterparts required precise alignment of positions, rotations, and fields of view. Video synchronization: Four independent recordings needed to be aligned so that events occurred at the correct moment across every feed. Dynamic reconstruction: Reconstructing moving people from real video was significantly harder than rendering a static LiDAR-scanned room. AI agent integration: Connecting Nemotron to meaningful investigation tools required coordinating structured evidence, tool calls, and report generation rather than simply building a chatbot. Evidence reliability: We needed to distinguish directly observed footage from reconstructed positions and AI-generated interpretations. Limited development time: We prioritized a working end-to-end experience connecting real CCTV footage, spatial replay, and autonomous AI investigation. Accomplishments that we're proud of We're proud that Sentry4D brings together real-world spatial reconstruction and working agentic AI in one application. We built: • A real LiDAR-based 3D digital twin. • Four synchronized camera feeds. • Free-roam spatial investigation. • Camera calibration and 3D person tracking. • X-ray visualization. • A working NVIDIA Nemotron investigation agent. • Multi-step AI workflows with integrated evidence-retrieval tools. • AI-generated incident reports connected to the 3D timeline. We didn't just build another surveillance dashboard. We built a system where AI can investigate an incident and humans can explore the evidence in 3D.
What we learned
We learned that computer vision becomes much more powerful when combined with spatial computing and autonomous AI. Building Sentry4D taught us about LiDAR reconstruction, multi-camera geometry, synchronization, person tracking, 3D rendering, and NVIDIA Nemotron tool integration. We also learned that an AI investigation system should do more than generate plausible explanations. It needs to retrieve evidence, distinguish observations from estimates, and make its findings verifiable. The biggest lesson was that the future of security lies in combining spatial understanding with intelligent automated investigation. What's next for Sentry Cam 3D Infinite Virtual Cameras: Reconstruct incidents from arbitrary viewpoints, even where no physical camera exists. Photorealistic 4D Reconstruction: Use advanced neural rendering and 4D Gaussian Splatting to recreate people and movements with greater visual realism. Real-Time Nemotron Investigation: Extend our working AI agent to continuously monitor live camera networks and investigate events as they happen. Autonomous Security Workflows: Allow AI agents to identify incidents, cross-reference evidence, create reports, and recommend appropriate responses. Legacy CCTV Modernization: Bring advanced spatial intelligence to existing surveillance systems without requiring organizations to replace their cameras. Building-Scale Digital Twins: Expand from individual rooms to warehouses, campuses, airports, and entire facilities. Our mission is to create a future where investigators aren't restricted by camera placement, disconnected footage, or hours of manual review. The future of security isn't more cameras. It's infinite perspectives, powered by intelligent AI. Sentry4D — Any Angle. Any Moment. One Complete Picture.
