GuardianPaw

An ESP32 quadruped where deterministic edge safety and Strands-based AI reasoning work together.

Comment

GuardianPaw

GuardianPaw is an embodied home-safety agent that combines a physical ESP32 quadruped robot, deterministic edge safety, and a Strands-based high-level decision agent.

Small home robots are useful only if people do not have to constantly supervise them. But safety-critical behavior should not depend entirely on a language model. GuardianPaw explores a layered architecture where immediate safety stays deterministic on the robot, while an AI agent interprets state, recommends higher-level behavior, and escalates to a human when needed.

Inspiration

Many small robots treat motion control, sensing, and AI reasoning as one system. That makes it difficult to define what should happen when sensors fail or when an AI model makes an uncertain decision.

I wanted GuardianPaw to follow a different principle:

AI reasoning should sit above deterministic safety, not replace it.

The robot should handle immediate safety locally, while the agent reasons about what the robot is experiencing and decides whether autonomous operation should continue.

What it does

GuardianPaw currently combines three layers:

1. Physical quadruped

A real ESP32-based quadruped provides the locomotion platform.

Verified physical capabilities include:

  • Forward locomotion
  • Turning
  • Wi-Fi / HTTP control
  • Existing quadruped gait control

2. Deterministic edge safety

The ESP32 runs an obstacle-avoidance state machine:

CLEAR → OBSTACLE_DETECTED → BACKING_UP → TURNING → RECOVERING → CLEAR

For the current Phase 1-A prototype, ToF distance telemetry is simulated while the state machine runs on the real ESP32.

A Dry Run mode prevents the obstacle-avoidance controller from automatically driving the servos during software verification.

Invalid distance data or sensor faults fail safe and prevent autonomous forward movement.

3. Guardian Agent

Robot telemetry is represented as structured RobotState data and passed through a deterministic safety policy before being interpreted by a real Strands Agent.

The agent uses a custom safety tool:

evaluate_safety_policy

and returns validated Pydantic structured output including:

  • risk level
  • whether autonomous patrol is allowed
  • whether human intervention is required
  • recommended next action

The language model cannot relax the deterministic safety constraints.

Live Strands inference has been verified locally using Ollama with llama3.1.

The demo covers four scenarios:

  1. Normal Patrol
  2. Obstacle Detected
  3. Recovery
  4. Sensor Fault

In the Sensor Fault scenario, GuardianPaw disables autonomous patrol and requests human inspection.

How we built it

The robot layer uses an ESP32 and an existing quadruped locomotion baseline. GuardianPaw adds a distance-sensor abstraction, simulated ToF pipeline, obstacle-avoidance state machine, fail-safe behavior, and Dry Run actuator isolation.

The agent layer is written in Python using the Strands Agents SDK.

The decision pipeline is:

RobotState → Deterministic Safety Policy → Strands Agent → evaluate_safety_policy → Pydantic Structured Output → Deterministic Post-Enforcement

The default cloud provider integration uses Amazon Bedrock. Because AWS account activation was still pending during the hackathon, I also implemented an explicit local Ollama provider fallback.

Live Strands inference was successfully verified with Ollama + llama3.1.

The local fallback does not bypass any safety logic. Both provider paths use the same deterministic policy and structured decision model.

The repository also includes regression tests, testing instructions, architecture documentation, simulation disclosure, and third-party attribution.

Challenges we ran into

The largest challenge was building around real hardware constraints while keeping the demo honest.

The quadruped experienced power-related brownout risk, and the first VL53L1X module was damaged during soldering. Rather than claiming unfinished hardware integration, I separated the sensing layer from the safety logic and created a simulated telemetry mode so the ESP32 state machine could still be verified safely.

AWS account activation also prevented live Bedrock inference before submission. To keep the Strands Agent fully runnable, I added an Ollama provider fallback and verified real local inference with llama3.1.

Another challenge was structured output reliability. The Ollama provider does not support forced ToolChoice, so GuardianPaw does not rely on the model alone for safety. Deterministic enforcement is applied before and after model reasoning.

Accomplishments that we're proud of

  • Built and demonstrated a real quadruped robot
  • Ran the Phase 1-A safety state machine on a real ESP32
  • Added Dry Run isolation so software could be tested without unsafe actuator movement
  • Built a real Strands Agent with a custom safety tool
  • Verified live local inference with Ollama + llama3.1
  • Validated structured Pydantic decisions
  • Ensured the language model cannot override deterministic safety rules
  • Passed the policy/provider regression test suite
  • Published the project as an open-source MIT-licensed repository
  • Clearly documented simulated, verified, and pending capabilities

What we learned

The biggest lesson was that embodied AI should not make every decision in the same place.

Fast, safety-critical reactions belong close to the hardware. Higher-level reasoning is useful for interpreting situations, explaining risk, and deciding when a human should become involved.

Simulation was also extremely valuable. It allowed the state machine and agent architecture to be tested before every hardware component was ready.

Most importantly, I learned that a good robotics demo should clearly distinguish between what is implemented, what is simulated, and what is planned.

What's next for GuardianPaw

The next step is to replace simulated ToF telemetry with a real VL53L1X sensor after the robot's power system is upgraded and verified.

Future phases include:

  • Real VL53L1X ranging
  • Physical autonomous obstacle avoidance
  • MPU6050-based posture awareness
  • Vision perception
  • Read-only telemetry transport between ESP32 and the Guardian Agent
  • Amazon Bedrock live inference after AWS account activation
  • Higher-level patrol and human-escalation behaviors

The long-term goal is a small home robot that can handle immediate safety locally, understand its operating state at a higher level, and ask for human help only when necessary.

Current verification boundary

Verified

  • Physical quadruped locomotion
  • ESP32 Wi-Fi / HTTP control
  • Phase 1-A safety state machine on real ESP32
  • Dry Run actuator isolation
  • Real Strands Agent execution
  • Ollama + llama3.1 live inference
  • Custom safety tool
  • Structured output
  • Deterministic safety enforcement

Simulated

  • Current ToF distance telemetry

Pending / not yet verified

  • Physical VL53L1X ranging
  • Physical autonomous obstacle avoidance
  • MPU6050 integration
  • Vision
  • Amazon Bedrock live inference

Built With

  • ai-agent
  • amazon-bedrock
  • arduino
  • c++
  • esp32
  • i2c
  • llama3.1
  • ollama
  • pydantic
  • python
  • robotics
  • strands-agents
Share this project:

Updates

Submission history