Paw React — Evidence-based reaction camera for dogs

• 

  From real-time detection to an evidence-based GPT-5.6 report.

Inspiration

Dog reaction videos are entertaining, but they rarely provide enough context to explain what changed. We wanted a camera that measures observable movement, shows its evidence, and communicates uncertainty instead of making unsupported claims.

What it does

Paw React is a mobile-first Flutter web app that records a dog's response to sound. On-device COCO-SSD provides real-time guidance for framing, lighting, sharpness, and camera stability.

After recording, it compares a pre-sound baseline with two post-sound observations and produces a reaction score, confidence level, limitations, and before, after1, and after2 evidence frames.

For signed-in users, only these frames and structured measurements—not the full video—are sent through AWS Lambda to GPT-5.6 Sol. If the AI request fails, the original local report remains available.

How we built it

The client uses Flutter Web, JavaScript, TensorFlow.js, and COCO-SSD. The backend uses AWS Cognito, API Gateway, Lambda, S3, DynamoDB, and Secrets Manager.

The OpenAI API key remains exclusively in Secrets Manager. Lambda validates the evidence and calls the OpenAI Responses API with store: false.

Codex helped us extend the existing project, design the evidence contract, implement tests, debug camera behavior, resolve AWS deployment failures, and prepare the production release.

Challenges we ran into

Our main challenges were inconsistent mobile camera and zoom behavior, recording microphone audio while playing sounds, separating dog movement from camera shake, handling multiple dogs, and preventing AI explanations from becoming behavioral or medical diagnoses.

Accomplishments that we're proud of

We built a complete recording-to-report workflow with real-time on-device detection, inspectable evidence frames, a protected GPT-5.6 route, and a reliable local fallback. The OpenAI API key never reaches the browser, and the full video is never sent to the model.

What we learned

Combining deterministic local measurements with a narrow GPT-5.6 explanation was more transparent than sending an entire video to a model. The most useful AI result is not the most confident one—it is the one that clearly shows its evidence and limitations.

What's next for Paw React — Evidence-based reaction camera for dogs

We plan to test across more devices, lighting conditions, dog sizes, and multi-dog scenes; improve evidence selection and video compatibility; add clearer data-retention controls; and consult animal-behavior and hearing-safety experts before expanding the sound library.

Built With

• amazon-cognito
• aws-lambda
• aws-secrets-manager
• coco-ssd
• codex
• flutter
• gpt-5.6-sol
• openai-responses-api
• tensorflow-js
• vercel