Private On-Device Triage Agent (Alexa+)

Inspiration

Freelancers and small business owners spend countless hours manually sifting through raw consultation notes, client inquiries, and meeting transcripts to extract action items and assign urgency. Standard cloud AI solutions require uploading sensitive, proprietary client data to third-party servers, raising strict privacy and data compliance concerns. I built this project to provide an automated, local-first alternative that processes confidential workflows entirely on-device.

What it does

The Private On-Device Lead & Note Triage Agent ingests unstructured client text offline, extracts structured action items, tags priority levels, and drafts response templates—ensuring zero sensitive data ever leaves the local environment. It converts chaotic meeting dumps into actionable, standardized JSON schemas without relying on external API infrastructure.

Mandatory Product Feedback (Build, Ship, Shape Requirements)

Tool / SDK Evaluated

• Primary Track Tool: Alexa+ SDK / Amazon Developer Ecosystem APIs
• System Integration: Leveraged for local event handling, audio/text stream routing, and prompt orchestration into our local agentic triage pipeline.

Specific Feedback Breakdown

• What Worked Well: The SDK handled asynchronous payload distribution smoothly. Event hooks allowed our supervisor agent to intercept incoming transcripts and route them cleanly without dropping execution context.
• What Needs Work: Error logging during local authentication loops could provide clearer detail. When payload validation fails on local endpoints, receiving explicit schema field error codes instead of generic runtime exceptions would save significant setup and debugging time.
• Onboarding Experience (Zero to "Hello World"): Initial setup took approximately 25 minutes. The quick-start documentation was functional, though adding explicit end-to-end JSON schema examples directly in the SDK quick-start guide would make initial integration much faster.
• Would You Build With It Again & Why? Yes. For local-first, privacy-focused agent workflows, having a reliable event routing framework makes structuring multi-agent systems straightforward and fast to deploy.

How we built it

• Orchestration & Workflow Logic: Formulated using low-code/no-code node architectures to define strict schema extraction rules, decision trees, and output parameters.
• Local Inference Execution: Powered on-device via quantized Gemma models for offline classification and JSON entity extraction.
• Data Schema: Automatically standardizes input text into structured fields: action_items, urgency_level, and follow_up_draft.

Challenges we ran into

Ensuring consistent, deterministic JSON schema compliance from quantized open-weight models running on local hardware can be challenging. I solved this by optimizing low-code node constraints and strict prompt formatting rules to guarantee predictable, structured outputs every time without relying on cloud-based compute.

What's next for Private On-Device Lead & Note Triage Agent

Expanding the local execution layer to integrate with voice-driven agentic skills (MCP) via Alexa+, enabling business owners to query daily action items and triage summaries entirely through voice while maintaining 100% on-device data privacy.

Built With

• ai-agents
• alexa
• gemma
• json
• no-code
• relevance-ai