AndonVoice AI

An Alexa+ powered manufacturing incident response system that turns operator voice reports into real-time Andon calls, team alerts, downtime tracking, and AI-assisted 4M1E analysis.

Comment

Inspiration

When a production line stops, every second matters. Operators are usually the first people who understand the situation, but reporting a problem may require navigating multiple screens, contacting different departments, and repeating the same information. This slows escalation and increases downtime.

AndonVoice AI was inspired by real manufacturing workflows where operators, leaders, maintenance teams, and quality teams must coordinate quickly while maintaining clear responsibility and traceability.

What it does

AndonVoice AI is a voice-enabled manufacturing incident response system. Operators can report problems using natural voice or text commands, while supervisors can ask situational questions such as:

  • “What is happening?”
  • “Which lines are stopped?”
  • “What is the problem on HLA-A?”
  • “Show active Andon calls.”

The system combines the command with live application context—including the selected production line, shift, workstation, user role, active incidents, and language preference—to generate a relevant response.

For write actions, AndonVoice AI:

  1. Detects the production line and problem category.
  2. Presents a preview of the intended action.
  3. Requires explicit human confirmation.
  4. Checks the user's role and authority.
  5. Prevents duplicate active calls.
  6. Records the action in the Andon workflow and audit trail.

Read-only commands can summarize stopped lines, active calls, downtime, incident status, and operational priorities without changing production data.

The application also provides 4M1E decision support for Man, Machine, Material, Method, and Environment. Amazon Bedrock can generate evidence-based hypotheses and containment questions when configured, while a deterministic fallback keeps the demonstration reliable.

How we built it

We built the user interface with React, TypeScript, and Vite. The backend uses Express and exposes REST services together with a Model Context Protocol server built using the official MCP SDK and Streamable HTTP transport.

The simulated Alexa+ experience supports browser speech recognition, bilingual English and Indonesian interactions, conversational follow-up questions, and synchronized application context.

Amazon Bedrock Runtime provides optional AI-assisted 4M1E analysis. Firebase can be used as a production data adapter, while a self-contained fictional dataset allows judges to test the public demo without credentials or access to real factory information.

The project includes automated tests, Docker support, Render deployment configuration, GitHub Actions, security headers, rate limiting, optional API authentication, and health checks.

Challenges we ran into

The main challenge was balancing conversational speed with operational safety. A voice assistant should reduce reporting time, but it must never silently create, modify, or close a production incident.

We therefore separated read commands from write commands. Every write operation requires human confirmation, role validation, and duplicate checking.

Another challenge was understanding conversational Indonesian. Short follow-up questions such as “lalu line mana lagi?” depend on the previous conversation. We added intent memory and situational context so the assistant can interpret these questions correctly.

We also had to make the demo reliable without exposing employer data, credentials, or production infrastructure. All public demonstration data is fictional.

Accomplishments that we're proud of

  • A shared incident workflow across voice, web dashboards, REST, and MCP.
  • Situational awareness based on active production conditions.
  • Bilingual English and Indonesian conversations.
  • Follow-up question support.
  • Human confirmation for every write action.
  • Duplicate incident prevention.
  • Role-controlled incident closure.
  • Five MCP tools for incident creation, status, downtime, updates, and 4M1E analysis.
  • A public demonstration that contains no confidential factory data.
  • Automated tests and reproducible deployment infrastructure.

What we learned

Conversational AI is most valuable on the shop floor when it reduces friction without hiding responsibility.

An AI assistant should help people understand the situation, structure information, and accelerate escalation. Operational authority must remain with qualified people, especially when an action can affect production data or equipment.

We also learned that true integration requires more than placing a chat interface inside an application. The assistant must understand the same language, selected line, shift, workstation, role, incidents, and workflow state as the main application.

What's next for AndonVoice AI

Our next steps are:

  • Validated integration with Alexa+ devices.
  • Secure identity federation between Alexa and application users.
  • Real-time event streaming from manufacturing systems.
  • Durable multi-tenant incident storage.
  • Expanded Indonesian manufacturing vocabulary.
  • Evaluation using anonymized shop-floor scenarios.
  • Integration with MES, maintenance, and production-monitoring systems.
  • Field validation before any safety-critical production use.

AndonVoice AI is currently a functional hackathon prototype and decision-support system. It is not a certified safety system and does not replace emergency controls or authorized shop-floor procedures.

Built With

Share this project:

Updates

Submission history