Inspiration
The Northeast region of India presents challenging terrain, rapidly changing weather conditions, dense vegetation, mountainous landscapes, and difficult-to-monitor border areas. Field personnel often need to make sense of large amounts of visual and situational information quickly.
We wanted to explore how multimodal AI could assist in turning raw terrain imagery and field observations into structured, easy-to-understand intelligence.
This led us to build SentinelNE — an AI-powered terrain and border intelligence analysis tool designed as a decision-support system for field operations. Instead of manually interpreting images and compiling observations, users can provide a terrain/satellite image along with optional situation-report information and receive a structured intelligence brief within seconds.
The project was built for the Hack4Brahma Hackathon, with the goal of demonstrating how modern AI can assist personnel working in challenging geographical environments.
What it does
SentinelNE analyzes terrain/satellite imagery and optional field situation reports using Google's Gemini multimodal AI.
The user can:
- Upload satellite, aerial, or terrain images.
- Provide additional field context through a situation report.
- Generate an AI-powered intelligence brief.
- Receive a structured terrain analysis.
- Get a patrol difficulty score.
- Identify potential anomalies in the provided information.
- Receive strategic recommendations.
- Estimate resource requirements.
- Get an overall risk classification: LOW, MEDIUM, HIGH, or CRITICAL.
- View the complete report directly in the application.
- Download the generated intelligence brief as a Markdown file.
The generated report follows a consistent 7-section format:
- Terrain Analysis
- Patrol Difficulty Score
- Anomaly Detection Report
- Strategic Recommendations
- Resource Requirements
- Risk Assessment
- Analyst Notes
This makes the output much easier to consume than an unstructured AI response.
How we built it
SentinelNE was built as a lightweight Python application using Streamlit for the interactive interface and Google Gemini as the multimodal AI analysis engine.
Technology Stack
- Python 3.10+ — Core application
- Streamlit — Web-based user interface
- Google Gemini 2.0 Flash — Multimodal AI analysis
- Pillow — Image processing and optimization
- python-dotenv — Secure API-key configuration
Application Flow
Terrain Image / Situation Report
↓
Image Processing
↓
Gemini Multimodal AI
↓
Structured Analysis
↓
Risk Extraction
↓
Intelligence Dashboard
↓
Downloadable Report
The application processes uploaded images before sending them to Gemini and converts them into an appropriate format for multimodal analysis.
We also designed a structured prompt that instructs Gemini to produce a consistent intelligence brief rather than a generic response. The application then extracts the overall risk level from the generated report and presents it visually in the interface.
The UI was designed around a dark, dashboard-style interface to make the application feel appropriate for an intelligence-analysis environment while keeping the workflow simple: provide data → analyze → review intelligence brief → download.
API credentials are kept outside the source code through environment variables, allowing the project to be run locally without exposing secrets in the repository.
Challenges we ran into
1. Making AI output structured and consistent
A major challenge was getting a general-purpose multimodal model to consistently produce useful information in a predictable format.
Instead of simply asking Gemini to "analyze this image", we designed a detailed structured prompt that defines exactly what sections the intelligence brief should contain.
This made the output significantly more usable inside the application.
2. Combining visual and textual information
Terrain imagery alone may not provide enough context.
For example, a field operator may know the weather conditions, recent activity, time of observation, or other situational details that cannot be inferred reliably from an image.
To solve this, SentinelNE accepts both:
- Visual information from the uploaded image
- Contextual information from the situation report
Both are passed to the multimodal model together.
3. Handling API limitations and rate limits
Working with a free-tier AI API introduced quota and rate-limit considerations.
We added error handling for:
- Invalid API keys
- API quota exhaustion
- Rate limits
- Safety filtering
- Model availability errors
- General API failures
This allows the application to provide understandable error messages rather than simply crashing.
4. Processing images efficiently
Large images can increase processing time and API usage.
We therefore introduced an image-processing layer that prepares uploaded images before sending them to the AI model.
5. Designing a useful interface around AI
Another challenge was avoiding a simple "chatbot" experience.
Instead, we designed SentinelNE as an intelligence dashboard, with dedicated input and output areas, visual risk classification, quick analysis statistics, structured reports, and downloadable results.
Accomplishments that we're proud of
We are particularly proud of turning a relatively simple AI API integration into a complete end-to-end application.
🛰️ Multimodal intelligence analysis
SentinelNE can reason over both images and textual field context, rather than relying solely on text prompts.
📋 Structured intelligence briefs
Instead of returning an unstructured AI response, the application generates a standardized seven-section report covering terrain, patrol difficulty, anomalies, recommendations, resources, risk, and analyst notes.
🚦 Automatic risk classification
The system extracts and displays an overall LOW / MEDIUM / HIGH / CRITICAL risk level from the generated intelligence report.
💻 Complete working interface
We built the entire workflow into a single Streamlit application, including image upload, situation-report input, analysis, visualization, error handling, and report downloading.
📥 Exportable reports
Generated intelligence briefs can be downloaded directly as Markdown files, making the output easy to save and share for demonstration and further review.
🔐 Practical API-key handling
The application uses environment variables for API credentials instead of hardcoding secrets into the source code.
Most importantly, we are proud that SentinelNE demonstrates a practical way of applying multimodal generative AI to geographical and situational analysis, rather than using AI only as a conversational interface.
What we learned
Building SentinelNE taught us that developing an AI application is much more than simply connecting an API to a user interface.
We learned how to:
- Work with multimodal AI models and image inputs.
- Design prompts that produce predictable, structured outputs.
- Combine visual data with contextual textual information.
- Process and optimize images before AI inference.
- Build interactive AI applications using Streamlit.
- Handle API failures, rate limits, quotas, and invalid credentials.
- Extract structured information from natural-language AI responses.
- Design an AI interface around a specific workflow rather than building a generic chatbot.
- Keep API credentials separate from application source code.
- Think about the limitations of AI-generated analysis and the importance of human verification.
One of our biggest takeaways was that AI should be treated as a decision-support tool rather than an unquestionable source of truth. The application therefore clearly communicates that generated reports need to be verified against ground truth before being used for real-world decisions.
What's next for SentinelNE
SentinelNE is currently a hackathon prototype, but there is significant room to evolve it into a more robust intelligence-support platform.
🌐 Geospatial integration
Integrate real geospatial data, satellite imagery sources, GPS coordinates, elevation data, weather information, and digital elevation models instead of relying only on manually uploaded images.
🗺️ Interactive mapping
Introduce an interactive map interface where users can select an area of interest and combine geographical information with AI-generated analysis.
📊 Historical intelligence
Store previous reports and observations to identify changes in terrain, activity, weather, or other relevant patterns over time.
🤖 Multi-model analysis
Use multiple specialized AI models or analysis pipelines for different tasks such as terrain classification, object detection, anomaly detection, and report generation.
📱 Field-ready deployment
Develop a lightweight mobile-friendly interface that can be used in low-connectivity environments and potentially support offline preprocessing.
🔄 Continuous intelligence
Move from one-time image analysis toward a system capable of continuously ingesting new observations and highlighting significant changes.
👤 Human-in-the-loop verification
Introduce mechanisms for trained personnel to verify, correct, and annotate AI-generated findings, improving reliability and creating feedback for future analysis.
🔐 Production-grade security
For real-world deployment, the system would require stronger authentication, authorization, encryption, audit logging, secure infrastructure, access controls, and strict data-governance policies.
Ultimately, our vision for SentinelNE is to evolve from a hackathon demonstration into a responsible AI-powered geospatial decision-support platform, where AI helps humans process complex information faster while keeping final decisions firmly under human control.
Built With
- ai
- analysis
- api
- computer
- data
- gemini
- generative
- geospatial
- image
- imagery
- intelligence
- language
- learning
- machine
- multimodal
- natural
- processing
- python
- satellite
- streamlit
- terrain
- vision
Log in or sign up for Devpost to join the conversation.