Inspiration
In a hazardous-material incident, the first few minutes are critical but chaotic. Network connections drop, and workers wearing heavy PPE (Hazmat suits, thick gloves) struggle to type on screens. This led me to a fundamental question:
Can we build a tool that helps operators make safer first decisions without pretending to know more than the evidence supports?
Sentinel_X is my answer: an offline-first, human-in-the-loop decision-support system. It empowers field workers to report incidents via speech-to-text, turns those observations into structured evidence, retrieves deterministic guidance, and records the complete decision trail—all without internet access.
What it does
Sentinel_X replaces fragile cloud dependencies with a deliberately constrained, inspectable safety pipeline.
For a high-consequence first-response path, I designed the system around this core philosophy:
$$ \text{Operational trust} \approx \text{availability} + \text{explainability} + \text{human control} $$
Key Features:
- Hands-Free Voice Input: Operators can speak their observations. The system uses a local STT engine to instantly transcribe audio, bypassing the need to type with thick gloves.
- Evidence Outranks Claims: If an operator says a spill is "just water," but the system detects a "corrosive" label in the report, it flags a critical conflict and escalates to a human.
- Conservative Degradation: When container identities are unknown, it doesn't hallucinate. It falls back to an "unknown-hazard" SOP and recommends the highest level of protection.
- A Real Safety Gate: It provides guidance but explicitly blocks critical physical actions (e.g., valve operation, shutdown) until an authorized human approves them.
How I built it
Although Sentinel_X is a 100% offline runtime, its creation heavily relied on OpenAI's latest models:
Codex: I used Codex to write 100% of the core architecture, including the FastAPI backend, the Streamlit frontend, and the complex logic for the local faster-whisper integration.
GPT-5.5: I leveraged GPT-5.6 during the design phase to synthesize complex Hazmat protocols and generate the deterministic JSON SOP knowledge base that the system queries locally. I built the MVP in Python with a FastAPI backend and a Streamlit interface. The system intentionally avoids OpenAI APIs, LangChain, or vector databases to guarantee 100% offline availability. And GPT-5.6 for testing code.
The architecture flows through these local components:
- Audio Processing: A highly optimized, local
faster-whisperCPU model (tiny.en) handles zero-latency speech-to-text. - Evidence Observer: Extracts UN codes, hazard symbols, and keywords from the text.
- Deterministic Retriever: Matches findings against local JSON SOP records.
- Risk Policy & Audit Memory: Assigns risk, enforces human approval, and logs every step.
To ensure safety, I engineered a strict, hard-coded confidence policy rather than relying on an LLM's self-evaluation. A single supported UN code provides strong evidence, while keywords alone are weaker:
$$ \text{confidence} = \begin{cases} 0.90, & \text{one supported UN code is detected} \ 0.70, & \text{multiple supported UN codes are detected} \ 0.60, & \text{only hazard keywords are detected} \ 0.20, & \text{no supported evidence is detected} \end{cases} $$
Recommendations with a confidence score below $0.85$ strictly require human confirmation.
Challenges I ran into
- Resisting AI Hallucination: In a safety context, a polished but unsupported answer is lethal. The hardest challenge was forcing the system to explicitly say "I do not know" and degrade conservatively rather than guessing a chemical's identity.
- Handling Conflicting Evidence: A simple keyword matcher can find both reassuring claims and hazardous labels. I had to build a policy to escalate conflicts rather than average the risk.
- Offline Constraints: Integrating an STT model that is fast and accurate enough for industrial use, yet small enough to run on a local CPU without external API calls.
Accomplishments that I'm proud of
- 100% Offline-First Architecture: Proving that critical safety software doesn't need to rely on unpredictable cloud APIs or internet connectivity to be intelligent and effective.
- Seamless Voice Integration: Successfully embedding the local
faster-whispermodel to bridge the gap between complex AI voice processing and practical, zero-latency field utility for workers in heavy PPE. - The Safety Gate Design: Demonstrating a practical and programmatic way to balance automated decision support with mandatory human oversight and complete auditability.
What I learned
I learned that innovation in safety software is not always about adding more automation—sometimes it is about choosing exactly where automation must stop. The most valuable design principles I discovered were:
- Make uncertainty visible and actionable.
- Prefer hard evidence over conversational tone.
- Keep critical operational decisions with qualified humans.
- Record the reasoning, not just the final recommendation.
What's next for Sentinel_X: Offline Hazmat AI Agent
The immediate next steps are to expand the local JSON knowledge base with more UN codes and biological threat profiles, implement role-based approval workflows, and add audio-filtering pre-processing for extremely noisy industrial environments.
In one sentence: Sentinel_X is an offline, explainable, human-controlled decision-support system that turns chaotic incident reports into conservative, auditable first-response guidance.
Log in or sign up for Devpost to join the conversation.