FrontDoor Concierge Auto Porch Security & Alexa MCP Agent

AI-powered Ring doorbell concierge with Amazon Bedrock vision, self-hosted Alexa+ Model Context Protocol (MCP) server, and courier lockbox access control.

Comment

Inspiration

💡 Inspiration

We’ve all had that sinking feeling: you’re stuck at work or away from home, your phone buzzes with a generic doorbell notification, and you open the video feed just in time to watch a courier balance your fragile delivery on a wet porch railing—or worse, an opportunistic porch pirate walking away with it.

Traditional smart doorbells are completely passive. They act as mere witnesses to theft or bad weather, alerting you after the damage is already done. Shouting through laggy two-way phone audio rarely helps, and couriers on tight schedules don't have time to wait for you to pick up.

We built FrontDoor Concierge to answer a simple question: What if your front door had an attentive, courteous digital doorman? Instead of an annoying buzzer, we envisioned an autonomous guardian that can see, think, and talk in real time—recognizing couriers by uniform, politely guiding them to a secure parcel box with a dynamic one-time PIN, and defending your home against porch piracy before you even unlock your phone.

🚀 What It Does

FrontDoor Concierge turns your porch into an intelligent, autonomous physical security hub:

  1. Multimodal Vision & Courier Recognition: Intercepts doorbell rings and camera snapshots, sending them to Amazon Bedrock (Claude 3.5 Sonnet) to classify courier uniforms (Amazon Logistics, UPS, FedEx, USPS, DHL), assess parcel dimensions, and compute a live risk score (0–10).
  2. Autonomous Two-Way Voice Concierge: Greets couriers instantly through the doorbell speaker (e.g., "Thank you for the Amazon delivery! Please enter PIN 4821 to store it in the porch lockbox").
  3. Self-Hosted Alexa+ MCP Server (Spec 2025-11-25): Conforms to the Model Context Protocol standard over Streamable HTTP. Homeowners can ask Alexa:
    • "Alexa, who is at the front door?"
    • "Alexa, was my Amazon package safely delivered?"
    • "Alexa, generate a PIN for the UPS driver"
    • "Alexa, sound the porch siren"
  4. Interactive Virtual Smart Lock Keypad: Issues time-decaying 4-digit one-time PINs for delivery couriers to unlock the porch parcel security compartment, protected against replay attacks.
  5. Multi-Camera Fleet HUD: Real-time switching between Front Door, Porch Lockbox, and Driveway cameras with Infrared Night Vision and live telemetry.
  6. Synthesized Web Audio Effects: Realistic synthesized doorbell chimes, 90dB warbling emergency sirens, smart lock mechanical hums, and Alexa wake tones with zero external MP3 dependencies.

🏗️ How We Built It

  • Hardware & Ring Ingestion: Ring developer ecosystem via ring-client-api with a zero-friction fallback simulator.
  • Multimodal AI Vision: Amazon Bedrock Runtime SDK (@aws-sdk/client-bedrock-runtime) utilizing anthropic.claude-3-5-sonnet-20241022-v2:0 for zero-shot image classification and risk reasoning.
  • Agent Standard: Self-hosted Model Context Protocol (MCP) server conforming to the November 2025 specification with Streamable HTTP JSON-RPC 2.0 and Server-Sent Events (/mcp/stream).
  • Backend Hub: Node.js & Express server with real-time WebSocket event broadcasting and atomic JSON persistence.
  • Frontend Dashboard: React 19 + Vite command center with dark glassmorphism styling, live scan-line animations, interactive JSON-RPC payload inspector, and Web Speech API audio synthesis.

🧗 Challenges We Ran Into (Friction Log)

We documented our developer friction in an official Friction Log (FRICTION_LOG.md) for the +10% judging bonus:

  1. Ring Headless 2FA Token Lifecycles: Solved by engineering an abstracted dual-mode adapter that guarantees continuous testability without requiring hardware re-authentication.
  2. MCP Streamable HTTP Transport vs. Stdio: Designed an Express JSON-RPC 2.0 router to make MCP accessible over remote HTTP and browsers.
  3. Bedrock Multimodal Base64 Overhead: Implemented in-flight downsampling and quantization to achieve sub-500ms inference.

📚 What We Learned

  • How to architect robust agentic workflows using the Model Context Protocol (MCP), connecting natural language intent to concrete smart home hardware actions.
  • The power of Amazon Bedrock's multimodal vision for real-time edge security and visual classification.

🔮 What's Next for FrontDoor Concierge

  • Integrating with Amazon Key for In-Garage/In-Gate delivery APIs.
  • Autonomous parcel weight sensors inside the physical lockbox to verify drop-off completion.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for FrontDoor Concierge Auto Porch Security & Alexa MCP Agent

Built With

Share this project:

Updates

Submission history