-
-
architectural_diagram
-
UI landing page
-
thumbnail
Inspiration
The inspiration for GeoSentinel came from the growing complexity of our global landscape. In an era of non-stop news cycles, shifting geopolitical boundaries, and thousands of live data signals, we realized that an analyst's biggest enemy isn't a lack of information—it's fragmentation. We wanted to create a "God's-Eye" terminal, similar to the high-density tactical interfaces seen in films or elite intelligence platforms like Palantir and Glint.trade, but powered by the intuitive, multimodal logic of Gemini 2.0 Flash.
What it does
GeoSentinel is a live, interactive 3D digital twin of Earth that fuses disparate OSINT (Open-Source Intelligence) streams into a single, unified viewport.
Visual Intelligence: It renders thousands of live points: Aviation arcs, Maritime paths, NASA FIRMS wildfire clusters, Infrastructure outages, and physical Conflict zones. Strategic Dashboard: Beyond the globe, a Command Center visualizes "hidden" signals, such as the Pentagon Pizza Index (predicting activity based on delivery trends) and AI-driven strategic posture ratings. Gemini Live Agent: At the core is the Omniscient Voice Agent. Users don't click buttons to find information; they talk to the globe. You can ask for sit-reps, command the camera to "Fly to [Location]," or request an AI-summarized executive brief—all through a low-latency, multimodal voice interface that handles natural interruptions.
How we built it
We architected GeoSentinel as a high-performance, cloud-native stack:
Frontend: Built with React and Vite, utilizing react-globe.gl and Three.js for hardware-accelerated 3D rendering of complex datasets. Backend / Agent: A Python FastAPI server orchestrates the intelligence layer. We used the Google GenAI SDK to establish a bidirectional WebSocket tunnel for Gemini 2.0 Flash, enabling real-time audio and JSON directive processing. Cloud Infrastructure: The entire stack is containerized with Docker and deployed to Google Cloud Run. We automated the infrastructure-as-code using custom Bash deployment scripts for bonus points and rapid iteration. Intelligence Fusion: We plumbed APIs from GDELT (news), USGS (earthquakes), OpenSky (aviation), and AIS (maritime) directly into the agent's spatial awareness. Challenges we ran into One of our primary hurdles was Multimodal Grounding. While Gemini is incredibly knowledgeable, geography is precise. We initially faced "coordinate hallucinations" where the agent tried to fly to locations it couldn't precisely pin. We solved this by implementing a Nominatim Geocoding Tool within the agent's logic, forcing it to validate every location name against OpenStreetMap before executing a camera move. Another challenge was Latency Synchronization. Handling "Live" interactions meant we had to ensure that when the agent spoke about a region, the UI reflected those specific satellite views and news articles instantly. This required a tightly coupled state machine between our WebSockets and the React state.
Accomplishments that we're proud of
We are particularly proud of the "Barge-In" (Natural Interruption) functionality. Achieving a natural conversation flow where a user can interrupt the AI mid-briefing to change the mission parameters (e.g., "Actually, skip that, show me Taiwan instead") makes the experience feel truly agentic rather than just a chatbot. We’re also proud of the UI Density—maintaining 60FPS while rendering thousands of live tracking arcs and pulsing conflict rings on a mobile-responsive 3D globe.
What we learned
We learned that multimodal AI is the death of the 'Text Box'. In high-stakes monitoring environments, nobody has time to type. The ability to "talk" to data and have the interface react spatially is a paradigm shift in UX. We also gained a deep appreciation for the Gemini Live API's ability to handle audio-to-JSON directives, effectively acting as both the ears and the brain of the application.
What's next for GeoSentinel
The future of GeoSentinel lies in Predictive OSINT. We plan to integrate historical trend analysis to allow the agent to move from "What is happening?" to "What will happen next?". Using the $\text{Tension Meter}$ math $T = \sum (Conflicts \times Gravity)$, we aim to automate the detection of emerging geopolitical flashpoints before they hit the headlines. We also intend to add Vertex AI Multimodal Vision capabilities, allowing the agent to "see" and describe live satellite imagery as it passes over the globe.
Log in or sign up for Devpost to join the conversation.