Homepage
Deep Topic
Pace of mind
Social

🐝 EXAMEN

Project Name: EXAMEN — Ambient Executive Intelligence
Tagline: Turning continuous ambient conversation from Bee Wearable AI into an executive second brain, orchestrated by AWS Bedrock AgentCore and served to Alexa+.
Live Demo: https://examen.devgiov.com
MCP Endpoint: https://examen.devgiov.com/mcp (FastMCP 2025-11-25 Streamable HTTP)
Target Tracks: Bee (Wearable AI) Track, Alexa+ Track, AWS Builder Mini-Challenge, open-source

💡 Inspiration: From Cognitive Overload to Invisible Assistance

Think about how many micro-interactions fill a single day:

A promise made to a coworker by the coffee machine ("I'll send you the migration repository by 5 PM").
An intense technical discussion in a morning sync ("We should evaluate AWS Bedrock AgentCore and LangGraph for containerized microVM agents").
A lingering domestic worry that strikes you 20 minutes after leaving home ("Did I unplug the clothes iron before locking the front door?").

Today’s voice assistants fail us in these moments because they are fundamentally passive and reactive. They force you to stop what you are doing, interrupt your social flow, utter an awkward wake-word, and articulate a rigid command: "Hey Assistant, set a reminder to send Dario the repo at 5 PM."

In the real world, human conversation is continuous, chaotic, and ambient. We asked ourselves:

What if your assistant was always listening in the background, filtered out 90% of everyday chatter with zero latency, and autonomously synthesized only the genuine executive signals into actionable intelligence?

That is the vision behind EXAMEN. Named from the Latin word for both a swarm of bees (honoring the Bee Wearable) and a careful, thorough examination of life’s signals, EXAMEN is an ambient executive second brain. It listens without wake-words, curates relational commitments, conducts autonomous web research on meeting topics, certifies physical action states, and makes everything queryable by voice on Alexa+ and touch on Echo Show.

🚀 What It Does: Four Ambient Primitives

EXAMEN operates as an autonomous background pipeline that ingests continuous audio from the Bee Wearable AI and enriches it across four key intelligence primitives:

1. 🧠 Deep Topic Event Engine & Autonomous Research

When users engage in technical, academic, or strategic meetings, EXAMEN doesn't just produce a flat transcript. Its R0–R9 state machine continuously tracks topic continuity, detects macro-discussion boundaries, and decouples events from arbitrary audio cuts.

Uses LLM to extract structured chapters, executive summaries, and actionable takeaways.
Automatically triggers a multi-tier autonomous web search tool to ground discussed technologies and attach verified official documentation links.
Formats executive dossiers ready for both desktop review and multimodal touch interaction on Echo Show 10/15.

2. 🤝 Bilateral Social Commitments (Relational Memory)

Spoken promises vanish quickly. EXAMEN recognizes relational commitments and parses them into explicit bilateral structures:

Debtor vs. Creditor tracking: It distinguishes between "I owe Dario the project documentation" (User is debtor) and "Dario owes me an architecture review" (User is creditor).
Auto-scheduled Daily Routines: Automatically provisions deadlines and converts verbal promises into tracked tasks within the user's daily agenda—with zero manual typing.

3. 🛡️ Certified Action-State Verification (Peace of Mind)

To eliminate autopilot anxiety, EXAMEN listens for domestic and operational confirmations:

Certifies actions such as "Clothes iron unplugged", "Front door locked", or "Car keys taken".
Records tamper-evident ISO timestamps alongside the verbatim transcription snippet, giving users immediate reassurance when they ask their Echo Show from the office: "Alexa, did I unplug the iron?".

4. 🎙️ Speaker Review & Privacy Guardrails

Ambient listening demands uncompromising privacy:

Any speech segment not attributed with high confidence to the registered user is quarantined into the Speaker Review Drawer.
The wearer can inspect untagged snippets and accept or dismiss them with a single click, ensuring pristine data provenance and preventing external chatter from polluting their personal knowledge graph.

🏗️ How We Built It: Architecture & Technologies

EXAMEN is engineered as a robust, production-grade cloud system designed for 24/7 reliability.

 [ Bee Wearable AI ] ➔ (Continuous SSE Stream: bee stream --json)
         │
         ▼
 [ Semantic Gatekeeper ] (AWS Bedrock: Amazon Nova Micro — >90% noise dropped)
         │  (10% Executive Signal)
         ▼
 [ AgentCore Pipeline ] (LangGraph State Graph)
    ├── Deep Topic Research Node (Autonomous DuckDuckGo Search)
    ├── Bilateral Commitments Node (Debtor / Creditor Extraction)
    ├── Action-State Verifier Node (Timestamp Certification)
    └── Speaker Review Node (Provenance & Privacy Filter)
         │
         ▼
 [ Amazon DynamoDB ] (Multi-table Tenant Isolation: examen-user, examen-auth, examen-ops)
         │
         ├─────────────────────────────────────────┐
         ▼                                         ▼
 [ Next.js 14 Web Dashboard ]              [ FastMCP Server (22 Tools) ]
 (CloudFront + EC2 eu-central-1)          (Streamable HTTP for Alexa+ on AWS)

1. Ingestion & Semantic Gatekeeper (AWS Bedrock)

The Bee wearable streams live transcripts over Server-Sent Events (bee stream --json). Instead of passing raw audio to expensive models, our Semantic Gatekeeper runs on Amazon Nova Micro on AWS Bedrock. It evaluates every utterance in under $350\text{ ms}$ and drops over $90\%$ of ambient noise (casual banter, television, background chit-chat) at sub-cent cost.

2. Agentic Orchestration (AWS Bedrock AgentCore & LangGraph)

When an executive signal is identified, it enters a LangGraph state graph orchestrated through our AWS Bedrock AgentCore Runtime client (src/pipeline/remote.py). The pipeline delegates to specialized nodes:

Topic Extractor & Explanatory Synthesis.
Autonomous Web Grounding.
Social Profiler & Bilateral Promise Resolver.
Action-State Certifier.

3. Cloud Data Layer (Amazon DynamoDB Multi-Table Design)

Deployed in AWS eu-central-1 (Frankfurt), EXAMEN utilizes three isolated DynamoDB tables:

examen-user: Stores contacts, commitments, certified actions, topic dossiers, segments, and briefings, partitioned strictly by user_id with sub-10ms point queries.
examen-auth: Manages argon2id password hashes, sessions, and personal MCP access tokens.
examen-ops: Telemetry, background task execution logs, and incident tracking.
Semantic search across briefings is powered by Amazon Titan Text Embeddings V2 (1024 dimensions). (A fully compatible SQLite + FTS5 backend is also maintained for local development and homelab environments).

4. Alexa+ FastMCP Server (Streamable HTTP)

To deliver on the Alexa+ Hackathon Track, EXAMEN implements a production FastMCP Server compliant with the MCP Specification 2025-11-25 over Streamable HTTP, live on AWS at https://examen.devgiov.com/mcp. It exposes 22 active tools across four domains:

Actions & Verification: query_certified_actions, verify_action_state, create_action, edit_action, delete_action (2-step confirm).
Topics & Briefings: get_briefing_report, search_deep_topics, list_topic_segments, query_topic_sources.
Social & Commitments: list_commitments, get_contact_dossier, get_upcoming_tasks, update_relationship_notes.
Speaker Review & Administration: list_items_to_review, accept_item, dismiss_item, get_system_telemetry.

5. Web UI & Echo Show Multimodal Frontend

Built with Next.js 14, React 18, Tailwind CSS, and SWR. It connects via SSE to live event streams and features an interactive Echo Show 10/15 Preview Modal, giving users an instant visual rendering of how Alexa displays executive briefings.

💰 Math & Economic Efficiency

Continuous ambient audio is notoriously expensive if implemented naively. Sending $24\text{ hours}$ of raw audio transcripts directly to frontier models would cost over $\$12.50\text{ per day}$:

$$\text{Cost}{\text{naive}} = \frac{86{,}400\text{ s}}{15\text{ s / utterance}} \times 200\text{ tokens} \times \text{Rate}{\text{frontier}} \approx \$12.50\text{ / day}$$

EXAMEN solves this using a two-tier probabilistic filter:

$$\mathbb{E}[\text{Cost}] = N \cdot \Big( C_{\text{Nova Micro}} + (1 - P_{\text{filter}}) \cdot C_{\text{Frontier llm}} \Big)$$

Where:

$P_{\text{filter}} \ge 0.90$ (empirical drop rate on daily speech).
$C_{\text{Nova Micro}} \approx \$0.000035$ per screening call.
$C_{\text{Frontier llm}}$ is only incurred when genuine executive signals are isolated.

Real-world benchmark result: Across our standard evaluation suite of 25 diverse real-world scenarios, the total inference cost was exactly $\$0.0084$—less than one single cent per active day!

⚡ Challenges We Ran Into

At-Most-Once Delivery on Hardware Streams:
The real Bee wearable stream (bee stream --json) operates in real-time with at-most-once delivery. If the server process restarts or the network blips, spoken events are permanently lost. To solve this without user intervention, we built the Historical Transcript Backfill Engine: a virtual-clock background service that cross-references Bee's cloud conversation ledger, detects temporal gaps, and replays missing speech through the pipeline with de-duplication guards.
The "Topic Boundary" Problem in Fluid Conversation:
In natural meetings, people drift between topics, banter, and deep technical details. Fixed time windows (e.g. slicing audio every 5 minutes) fractured context. We designed the Deep Topic Event Engine (R0–R9 state machine), decoupling synthesis from Bee conversation UUIDs and using Amazon Titan V2 embeddings to compute cosine distance across consecutive topic segments, intelligently merging micro-discussions while triggering segment and final synthesis checkpoints.
Multi-Tenant Cloud Security:
Ensuring that voice-derived data remains strictly private was paramount. We engineered a fail-closed tenant_scope context architecture: DynamoDB queries enforce strict partition key boundaries (user_id), web sessions use HttpOnly SameSite=Lax Secure cookies, and Alexa+ access is gated by scoped Bearer tokens.
FastMCP Protocol over Cloud Reverse Proxies:
FastMCP's built-in DNS-rebinding protection initially rejected requests routed through CloudFront and NGINX with 421 Misdirected Request. We traced and resolved the host verification layer, validating the Streamable HTTP transport for production compliance.

🏆 Accomplishments We're Proud Of

588 Automated Tests (100% Green): A comprehensive test suite across 31 test files validating both SQLite and Amazon DynamoDB storage, the event engine, speaker review, authentication, and remote AgentCore clients.
Production AWS Deployment: Live and accessible at https://examen.devgiov.com on CloudFront, EC2 t3a.small, and Amazon DynamoDB in Frankfurt (eu-central-1).
22 Working MCP Tools: Fully conformant with the MCP 2025-11-25 Streamable HTTP specification for Alexa+.
Sub-Cent Cost Efficiency: Proven $0.0084 per session operational cost via two-tier AWS Bedrock model routing.
Transparent Developer Diary: A 13-entry FRICTION_LOG.md documenting platform insights, SDK quirks, and architectural workarounds.

🧠 What We Learned

Ambient Voice Requires Context-Aware Negative Filtering: Standard LLMs want to be helpful and tend to over-extract; when fed casual banter, they hallucinate tasks. Calibrating the Semantic Gatekeeper with few-shot ambient negative examples was crucial to achieving high precision.
Micro-Events vs. Macro-Narratives: Humans do not speak in database records. Creating intermediate data structures like SessionDossier and micro-event linking is essential to bridging real-time audio with actionable relational knowledge.
The Power of FastMCP for Voice Assistants: Exposing typed, schema-validated tools over Streamable HTTP transforms Alexa from a basic question-answering assistant into an intelligent executive interface capable of orchestrating complex backend graphs.

🔮 What's Next for EXAMEN

Official Alexa+ Store Add-on: Publishing our pre-architected Add-on package with OAuth 2.1 account linking once third-party developer access to @alexa-ai/cli is enabled post-hackathon.
Multimodal Visual Grounding: Combining the Bee microphone with wearable camera snapshots to visually confirm certified actions (e.g. visually confirming the locked deadbolt).