HATI - Homestead Autonomous Threat Intervention

HATI set up to watch my coop
HATI receiving feedback from my walkthrough
Telegram output of positive event
HATI activating the mister (red) from video of raccoon
Arming the diffuser
Uncertain event elevated to Luna
Certain event handled locally
Kids and chickens correctly identified locally

Inspiration

We live on some acres outside of the metro area, and we absolutely love our hobby farm. That said, there are some parts of farming that are not-so-great, and one of the absolute worst for me is telling the kids that "something got one" of their pet poultry. Predation is a constant threat, and keepers will use sturdy fencing, lights, noise, and even scent to ward off unwelcome guests to the coop. I have sturdy hardware cloth, motion sensing lights, and automatic closing doors. Unfortunately, predators are smart and will test the edges of deterrents, or become habituated to their presence.

Although I'm somewhat technical, I've never written code. I've also never entered a hackathon. So when I saw the ad said to bring ChatGPT and Codex your "most ambitious work," I thought: Sure, worth a shot. I told Sol I wanted to think of a hackathon project and then said "I lose chickens to predators. I have extra IP cameras. I could get automatic sprayers. I have a large bottle of [predator scent] concentrate." And, it colorfully knew exactly what I was thinking.

What it does

HATI stands for Homestead Autonomous Threat Intervention. A camera watches in front of the coop. When it sees a change, local motion detection captures five frames. A local Gemma image model suppresses unnecessary paid inference only when clearly benign resident-bird or human is the subject; it can never authorize hardware. Everything else goes to GPT-5.6 Luna in one structured vision request.

The model describes what it sees, but ordinary deterministic Python code decides whether action is permitted. HATI requires temporal predator agreement, checks a target allowlist and protected zone, and applies an absolute human veto. Unknown or incomplete evidence means no autonomous action.

An authorized event can trigger a full-mist diffuser deployment of scent for no more than five minutes. HATI confirms the active state, always attempts shutdown, verifies the off state, durably reserves each actuator attempt before dispatch, and reconstructs cooldown after restart so a crash cannot repeat a physical command.

Telegram gives the evidence and outcome, collects one-tap feedback, reports status, and supports safe testing. Correct feedback becomes protected regression evidence. A reviewed false alarm may propose only a more conservative classifier candidate, which is promoted only after correcting the miss with zero protected-case regressions.

How we built it

Python 3.12, OpenCV motion detection, typed event records, inspectable JSON traces, and 95 automated tests.
Low-latency Foscam RTSP capture with authenticated fallback and DHCP rediscovery.
GPT-5.6 Luna with five high-detail frames, structured output, one request per event, and token accounting.
Deterministic consensus, target allowlists, protected-zone checks, and an absolute human veto outside the model.
A loopback-only Gemma 4 E4B gate with bounded clear, likely, and uncertain output. Its first shadow run exposed an unsafe raccoon-as-chicken miss; removing a poultry hint corrected the protected rerun. Gemma may suppress benign calls but can never authorize deterrence.
Local Tuya control with full mist, a medium-blue status light, a five-minute cap, active-state confirmation, unconditional shutdown attempts, and verified off state.
Telegram alerts, owner feedback, status, test commands, and restart-safe orchestration.
A public interactive judge site and GitHub Actions CI.

Where Codex accelerated the work

Codex converted plain-language decisions into architecture, implementation, tests, operator scripts, documentation, and the judge experience. It helped diagnose an old camera whose credentials and Wi-Fi behavior were temperamental, recovered authenticated images, and replaced a 19-second JPEG capture path with a continuously drained RTSP stream. It mapped the consumer diffuser from water-only observations and turned each red-team question into a regression test. GPT-5.6 also helped me re-wire a cable after a critter chewed through it.

The important decisions were collaborative. I decided what was humane, useful, and honest in a homestead context. Codex made those boundaries executable. GPT-5.6 powers Luna's visual interpretation, while deterministic code keeps the model outside the physical authorization boundary.

Accomplishments

A real outdoor owner-pass event completed motion capture, one five-image GPT-5.6 request, deterministic HUMAN_VETO, a Telegram photo alert, and stored owner feedback.
The physical diffuser completed bounded tests and was verified off afterward.
The four-case judge demo runs without a camera, API key, network, or hardware.
A controlled improvement fixture moved from 3/4 to 4/4 with zero regressions. This proves the promotion gate, not field accuracy.
A non-coder and Codex produced a public, documented, tested hardware prototype during one build week.
I had Codex also design a new cover for the diffuser.

Challenges and honesty

The camera was old, DHCP changed its address, outdoor Wi-Fi was temperamental, and the consumer diffuser had no project-specific API. One cable had to be repaired thanks to a rodent, and I redid my wifi because it doesn't normally reach the barn area. Live wildlife also refuses to respect hackathon demo schedules. Wildlife replay and plush scenarios are explicitly labeled controlled; I declined to bait a raccoon for a hackathon. HATI has not claimed a live wildlife detection. A longer unattended field trial and broader owner-reviewed night set are future work.

What I learned

I learned that Codex was generally able to handle the coding aspect with little instruction from me, but it was always beneficial to question the conditions, actions, edge cases, and refinement pathways so that they could be made more robust. It was important that I asked a lot of questions, for example because Luna is more robust than my local model I was able to cut the API costs by having fewer images queried.

I also learned that Codex was very conservative on security and safety. I'm much less worried of an accidental or malicious deployment, considering I had originally planned to spray the perimeter from the start.