Atlas Sentinel

Physical AI that sees infrastructure, reasons over visual and sensor evidence, takes safe action, and verifies the real-world outcome.

Comment

Inspiration

Critical infrastructure often gives us warning signs before failure: a leak, abnormal flow, a damaged component, an unexpected obstruction, or a sensor drifting out of normal conditions. Yet these signals are usually fragmented across cameras, sensors, dashboards, and human operators.

We wanted to build something beyond “AI that detects.”

Atlas Sentinel gives physical infrastructure eyes, judgment, safe agency, and proof.

The core idea is a closed loop:

SEE → REASON → ACT → VERIFY → LEARN

Instead of stopping when computer vision identifies an anomaly, Atlas Sentinel uses visual evidence together with sensor and operational context to determine what matters, decide what should happen next, execute approved actions, and return to the physical scene to verify the outcome.

What it does

Atlas Sentinel is a Physical AI system for intelligent infrastructure monitoring and intervention.

In our smart water infrastructure scenario, a camera continuously observes the system while sensors provide operational telemetry such as water level and flow.

When an anomaly occurs, Atlas Sentinel:

  1. Uses OpenCV 5 to detect and analyze visual changes.
  2. Correlates visual evidence with sensor and asset context.
  3. Determines the most likely failure and its severity.
  4. Checks whether the proposed intervention is permitted by policy.
  5. Recommends or executes an appropriate action depending on risk.
  6. Uses human approval for consequential operations.
  7. Re-inspects the physical system after the action.
  8. Verifies that the expected real-world change actually occurred.
  9. Records the incident, evidence, action, and verified outcome.

For example, a visible pipe leak combined with abnormal flow can result in a high-confidence recommendation to isolate the affected section. After the valve is closed, Atlas Sentinel checks both the camera feed and telemetry to determine whether the leak signature and flow anomaly have actually decreased.

The system also knows when not to act. If the camera becomes obstructed or visual confidence falls below a safe threshold, autonomous intervention is disabled and additional inspection or human review is requested.

How we built it

Atlas Sentinel combines edge computer vision, sensor telemetry, cloud infrastructure, agentic orchestration, policy controls, and physical actuation.

Vision: OpenCV 5 performs camera calibration, frame analysis, motion and anomaly detection, spatial analysis, and temporal comparison. Visual observations are converted into structured evidence that becomes an input to the reasoning pipeline.

Physical layer: A camera, water-level and flow sensors, pump, valve, and embedded controller represent a miniature infrastructure node.

Cloud layer: AWS services provide device connectivity, event-driven processing, storage, operational state, observability, and workflow orchestration. The architecture is intentionally modular so workloads can move between edge and cloud depending on latency and reliability requirements.

Agentic layer: Atlas Sentinel separates observation from interpretation, authorization, action, and verification. The agent receives structured evidence, evaluates risk and policy, generates an action proposal, and routes consequential operations through a human approval gate.

Verification layer: Commands are never treated as proof of success. The system separately observes the resulting physical state and checks whether the expected outcome actually occurred.

COOL optimization: We designed the computer-vision workload so it can be benchmarked across a baseline environment and an optimized AWS Graviton/Arm environment. We measure latency, throughput, utilization, and workload cost using reproducible experiments rather than assuming optimization benefits.

Challenges we ran into

The biggest challenge was moving from computer vision as a prediction system to computer vision as part of a decision-making system.

A detector can say “there is water here.” Infrastructure intelligence needs to ask much harder questions:

Is it actually a leak? How severe is it? What other evidence supports the conclusion? Is the system allowed to intervene? What is the safest reversible action? How do we know the action worked?

We also had to design around uncertainty. Physical environments are noisy. Cameras can be obscured, sensors can disagree, and visual evidence can be ambiguous. Rather than hiding those limitations, we made uncertainty a first-class component of the architecture.

Another challenge was balancing ambition with reliability. It would be easy to build a sprawling infrastructure platform. Instead, we focused on one complete vertical slice that demonstrates the full loop from perception to verified physical intervention.

Finally, benchmarking optimization fairly requires discipline. Our COOL comparison is designed around the same workload, measurable metrics, and reproducible conditions so performance claims come from evidence rather than marketing.

Accomplishments that we're proud of

We are proud that Atlas Sentinel goes beyond a conventional vision demo.

The system connects:

physical sensing → OpenCV perception → contextual reasoning → policy → action → physical verification

into one coherent loop.

We are especially proud of the distinction between:

Intent ≠ Command ≠ Observation ≠ Verification

A system deciding that a valve should be closed is not the same thing as successfully closing it. Atlas Sentinel explicitly separates those states and requires the physical world to provide evidence that the intended outcome occurred.

We are also proud of the graduated-autonomy model. Low-risk operations can be automated within predefined boundaries, while consequential operations require human approval.

Most importantly, the system can fail safely. When perception becomes unreliable, Atlas Sentinel does not manufacture certainty. It reduces autonomy and asks for more evidence.

That principle is central to the project:

Never act with confidence you have not earned.

What we learned

We learned that building Physical AI is fundamentally different from building an AI system that only produces answers.

The hard part is not just perception. It is creating a trustworthy bridge between perception and physical consequences.

We learned to treat visual output as evidence, not truth.

We learned that temporal information is often more valuable than a single frame because infrastructure problems are defined by how states change.

We learned that agentic systems become much more useful when tools return structured observations rather than unstructured descriptions.

We also learned that verification must be designed from the beginning. The question is not simply:

“Did the model make the right prediction?”

It is:

“Did the physical world change as expected, and can we prove it?”

Finally, we learned that responsible autonomy is not about making an agent act as much as possible. It is about knowing exactly when the agent should act, when it should ask, and when it should stop.

What's next for Atlas Sentinel

The smart water node is only the beginning.

Next, we want to expand Atlas Sentinel into a general infrastructure intelligence platform where the same perception-and-verification architecture can operate across water, energy, agriculture, buildings, factories, roads, and environmental systems.

We plan to introduce richer digital twins, additional visual failure modes, predictive maintenance, multi-camera reasoning, active perception, and larger fleets of connected assets.

At larger scale, Atlas Sentinel could move from asking:

“What is failing right now?”

to:

“What is most likely to fail next?”

and eventually:

“Which intervention will create the greatest improvement in reliability, safety, and resource efficiency?”

Every asset could develop a continuous evidence history:

observe → diagnose → prioritize → intervene → verify → learn

Our long-term vision is to build an intelligence layer for the physical world—one that does not merely describe reality, but can responsibly participate in improving it.

Built With

Share this project:

Updates

Submission history