AURA

Proactive memory that fights for your focus.

Comment

Project AURA — Proactive Memory That Fights For Your Focus

The Pitch

Every AI wearable today is obsessed with recording your entire life. They listen to you for eight hours, dump audio into the cloud, and hand you an 8-page transcript at 6:00 PM when your brain is already fried.

I built AURA because I don't need another post-mortem of my day. I need an assistant that shows up before things go sideways—one that prepares me five minutes before a high-stakes meeting, defends my physical room against distractions, and respects my privacy at the hardware level.

💡 What Inspired Me

A few months ago, I was testing out modern audio wearables. Within three days, two things became blindingly obvious:

  1. Nobody reads post-meeting audio summaries. By the time a 30-minute meeting is over, the damage of being unprepared is already done. What I actually needed was context before walking in: Who am I meeting? What did we leave hanging last time? What are the three questions I need to ask right now?
  2. 24/7 ambient recording feels creepy. Wearing a live microphone into a room makes friends, family, and coworkers uneasy. Even if a company promises "we don't save raw audio," software promises don't mean much when an API key leaks or an engineer pushes buggy firmware.
  3. The environment is the biggest productivity killer. While I was rushing to prep for calls, the smart TV in the living room was streaming, Slack was pinging, smart bulbs were blindingly white, and room temperatures were climbing.

I wanted to flip the entire paradigm on its head: Shift from reactive transcription to proactive defense. Instead of recording what you forgot, AURA anticipates what you're about to forget at $T-5$ minutes, and commands your surroundings to protect your focus.

⚙️ How I Built It

I wanted AURA to be a real, end-to-end ecosystem—not just a web mockup wrapped around an LLM API call. It spans custom hardware, watchOS, mobile bridges, an 11-service AWS cloud backend, and a deterministic edge policy engine.

1. The Wearables & The Edge

  • The AURA Orbit Pebble (firmware/): I designed a custom hardware pebble around the Nordic Semiconductor nRF5340 dual-core ARM Cortex-M33 running Zephyr RTOS in C. I paired it with a transflective Sharp Memory-in-Pixel display ($<15\,\mu\text{W}$ idle draw) so it's always readable outdoors without battery drain, and added a DRV2605L haptic driver for subtle wrist taps.
  • Apple Watch App (apple/AURAWatch/): Written in native SwiftUI for watchOS 10. It reads real-time heart rate and HRV directly via HealthKit to gauge cognitive strain, and renders glanceable whisper cards you can digest in under 2 seconds.
  • Mobile Bridge (mobile/): A React Native (Expo) companion that connects to the wearables over BLE 5.3. Crucially, it runs a local PII redactor on-device: it sanitizes names, emails, and secrets before anything leaves the phone.

2. The Cloud Intelligence Layer (11 AWS Services)

I defined the entire cloud stack in TypeScript using AWS CDK v2 (infra/lib/aura-cloud-intake-stack.ts):

  • Amazon Bedrock (Claude 3.5 Sonnet): Handles the deep $T-5$ context synthesis via the Converse API.
  • AWS Lambda (Arm64 Graviton): Serverless microservices (focus_guardian, apns_dispatcher, websocket_handler, control_plane) with structured JSON logging and strict data boundaries.
  • Amazon DynamoDB: Stores ephemeral session context and audit trails with an automatic 30-day Time-To-Live (TTL) so user data self-destructs.
  • Amazon EventBridge & SQS: Schedules the exact $T-5$ triggers, manages dead-letter queues, and handles asynchronous signal dispatch.
  • AWS KMS, WAF & Secrets Manager: Protects API Gateway with rate-limiting, encrypts all tables at rest, and isolates OAuth secrets.

3. The Multi-LLM Resilience Gateway

In a time-critical window, an LLM timeout is fatal. If my meeting is at 2:00 PM, a prep card arriving at 2:01 PM is useless. I built a dual-engine gateway:

  • Primary: Amazon Bedrock (Claude 3.5 Sonnet) — average latency $\approx 680\,\text{ms}$.
  • Secondary (Failover): IBM Watsonx (Granite 3.0 8B Instruct) — average latency $\approx 420\,\text{ms}$. If Bedrock returns a 5xx or throttles, the gateway trips a circuit breaker and fails over instantly.
  • Edge Heuristic: If the internet drops entirely, a local Python heuristic generates a basic briefing card in $< 5\,\text{ms}$.

4. Deterministic Local Policy Gate

I have a strict rule: LLMs advise; they never execute. The cloud Lambda outputs policy_required: true. It can recommend pausing a TV or setting Slack to focus mode, but only the local policy engine on the user's device (standalone_agent/policy_engine.py) has permission to run actions against the local network.

🧮 The Math & Logic Under the Hood

1. Biometric Stress & Cognitive Strain Index

Rather than guessing when someone is overwhelmed, the Apple Watch monitors Heart Rate Variability using the Root Mean Square of Successive Differences ($RMSSD$) between successive heartbeats ($RR$ intervals):

$$RMSSD = \sqrt{\frac{1}{N-1} \sum_{i=1}^{N-1} (RR_{i+1} - RR_i)^2}$$

We compute a continuous cognitive strain score $S_t \in [0, 1]$ relative to the user's baseline resting mean $\mu_{\text{rest}}$ and variance $\sigma_{\text{rest}}$:

$$z_t = \frac{\mu_{\text{rest}} - RMSSD_t}{\sigma_{\text{rest}}}, \quad S_t = \frac{1}{1 + e^{-\alpha (z_t - \theta)}}$$

When $S_t > 0.75$ right before a meeting, AURA automatically intensifies its focus defense—dimming room lights earlier and enforcing stricter notification silences.

2. Mathematical Anti-Hallucination Guard

I couldn't risk Claude hallucinating context right before an executive call. We assign verifiable IDs to incoming context: $E_{\text{provided}} = {e_1, e_2, \dots, e_m}$.

When the model returns its proposed citations $E_{\text{cited}}$, our Pydantic validation layer verifies strict set containment:

$$E_{\text{cited}} \subseteq E_{\text{provided}} \quad \text{where} \quad E_{\text{cited}} \neq \emptyset$$

If the model outputs even a single hallucinated citation ID:

$$\exists \, c \in E_{\text{cited}} \text{ such that } c \notin E_{\text{provided}} \implies \text{REJECT} \to \texttt{safe_fallback}()$$

It immediately dumps the model's output and drops back to a safe, deterministic summary.

3. Jittered Exponential Backoff

To avoid crashing downstream APIs during connection spikes, the resilience gateway uses truncated exponential backoff with full jitter:

$$t_{\text{backoff}} = \min\left(t_{\text{max}}, \; t_{\text{base}} \cdot 2^k\right) \cdot U(0, 1)$$

where $t_{\text{base}} = 200\,\text{ms}$, $t_{\text{max}} = 3000\,\text{ms}$, and $U(0, 1)$ is a uniform random float between $0$ and $1$.

4. 100% Cryptographic Reversibility (Undo Tokens)

Whenever AURA dims a light, silences Alexa, or pauses Fire TV, it issues a single-use HMAC token:

$$\tau_{\text{undo}} = \text{HMAC-SHA256}\Big(K_{\text{device}}, \; \text{action_id} \parallel \text{client_id} \parallel t_{\text{issued}}\Big)$$

If the user taps "Undo" on their wrist within 15 minutes ($t \le t_{\text{issued}} + 900\,\text{s}$), the action is immediately reversed.

🧗 Challenges I Faced

1. Solving the Hardware Privacy Paradox

I realized early on that software-based privacy toggles don't convince skeptics. If a microphone is controlled purely by software, a rogue firmware update could secretly listen in.
How I fixed it: On the AURA Orbit circuit board, I wired the amber privacy LED physically in series with the MEMS microphone bias power rail. This means the microphone cannot physically receive electrical current without the amber ring lighting up. It's an un-hackable guarantee enforced by electrical physics, not code.

2. Fighting Python's Reserved Keywords in CDK Lambdas

A funny but frustrating bug: I named my serverless directory lambda/. When writing the test suite with pytest, from lambda.focus_guardian import ... threw immediate SyntaxError: invalid syntax because lambda is a reserved keyword in Python!
Workaround: Rather than breaking CDK path contracts mid-development, I used importlib.import_module("lambda.focus_guardian") across the test harness. A humbling reminder that names matter.

3. WebSocket Handshake Limitations on Wearables

Embedded chips and browsers running WebSockets cannot inject custom Authorization HTTP headers during the initial $connect upgrade request. Standard API Gateway docs assume you can pass whatever headers you want.
How I fixed it: I rewrote the WebSocket connection Lambda to accept a dual-path handshake—either an HTTP header or a cryptographically signed query param (?auth_token=), strictly validating tokens before allowing the connection into the DynamoDB connection pool.

4. The Local PII Redactor Eating Real Tokens

My phone-number regex was aggressive. In early testing, it collided with numeric sequences inside long 64-character session tokens, mangling auth packets before they left the phone!
How I fixed it: I redesigned the redactor into a strict, multi-pass pipeline: high-entropy tokens and API keys are identified and preserved first, and only then do strict boundary-checked regex passes scrub phone numbers and email addresses.

5. Smart Home False Positives

Having your TV pause while your spouse is watching a movie in the other room is a quick way to get your project thrown in the trash.
How I fixed it: I implemented household isolation rules (blocked_household_mismatch), per-device consent toggles, and an automated kill switch: if more than $5\%$ of actions are undone in an hour, AURA stops automatic actuation entirely and asks for permission every time.

🧠 What I Learned

  1. Proactive context is 10x more valuable than retroactive summaries. Spending engineering effort on predicting needs 5 minutes in advance creates a dramatically better experience than processing gigabytes of raw data after the fact.
  2. Hard mathematical boundaries beat prompt engineering every time. You can prompt Claude all day to "please don't hallucinate," but until you enforce strict Pydantic models and set-containment checks in code, you don't have a reliable product.
  3. Hardware-enforced privacy builds instant trust. The moment people see an amber LED physically hardwired to the microphone circuit, their skepticism disappears.
  4. Cloud services should recommend; local devices must decide. Keeping execution keys on the edge device protects users from runaway cloud actions and keeps the human in total control.

🏆 What I'm Most Proud Of

  • 300+ Automated Tests Passing: A comprehensive test suite covering unit logic, contract schemas, and multi-LLM failovers without needing flaky cloud mocks.
  • A Real Multi-Device Stack: It's not a slide deck. We have working C firmware on Nordic silicon, a native SwiftUI watchOS app, a React Native mobile bridge, and an interactive web simulator on Vercel.
  • Zero Hallucinations, Zero Surveillance: Zero raw audio touches the cloud, every citation is mathematically verified, and all ephemeral context expires after 30 days.

Built With

Share this project:

Updates

Submission history