NexCord

A voice-first, human-in-the-loop incident commander that turns live-event disruptions into simulated, approval-gated actions and verifies the outcome through MCP tools.

Comment

Inspiration

Live events are made up of tightly connected resources: rooms, equipment, people, vendors, schedules, and attendee capacity. A single failure such as a projector going down before a final can trigger a chain of time-sensitive decisions. I wanted to explore how an AI agent could help an event operator coordinate a response without letting a model make unchecked operational changes. The central idea is to combine useful autonomy with a clear human approval boundary.

What it does

NexCord is an agentic incident-response prototype for live event operations. It gathers event and resource context, identifies the operational problem, retrieves relevant information, generates candidate response plans, simulates resource changes, and asks an operator to approve the selected plan. Approved actions are executed through an MCP tool server, after which NexCord verifies the resulting state and records the operational history. Recovery paths exist for supported failure cases.

The demonstrated scenario is a Robotics Final with a projector-failure incident. NexCord proposes allocating a backup projector, the operator approves the plan, the system executes the action, and the UI reports that execution and verification succeeded.

How we built it

The frontend uses Next.js, React, and TypeScript. The API and agent backend use Python, FastAPI, and LangGraph. PostgreSQL stores event operations data; pgvector supports knowledge retrieval. A separate MCP server exposes tools over Streamable HTTP, while a Python ML service uses PyTorch/torchvision for person-detection and occupancy-related experimentation. Docker Compose runs the services locally and on an AWS EC2 instance. The AWS deployment uses private Amazon RDS for PostgreSQL, Secrets Manager for deployment secrets, Systems Manager Session Manager for administration, and CloudWatch Logs, metric filters, and alarms for observability. Caddy and Let's Encrypt provide HTTPS for the hosted demo; DuckDNS provides a free subdomain with automatic refresh of the instance's changing IP.

The current LLM configuration uses Google Gemini. Amazon Bedrock is not used in this project.

Challenges we ran into

Building a multi-container application for a small CPU-only EC2 instance exposed dependency, deployment, networking, and observability issues. The initial ML image build pulled large CUDA-related packages despite the target machine being CPU-only; the AWS image definitions were changed to select PyTorch's CPU wheels explicitly and were checked for a CPU-only Torch build. A broad .gitignore rule also hid SQLAlchemy model files, which caused imports to fail in the deployed container; narrowing the rule and tracking the source modules fixed it. AWS Compose commands failed when the separate .env.aws file was not passed, and Docker commands in Systems Manager needed sudo because the SSM user did not have Docker-socket permissions. We also had to distinguish permissions available to the EC2 instance role from the permissions of the laptop's AWS CLI identity when updating the firewall. Finally, CloudWatch metric filters required structured Nginx logs to produce useful per-request latency data, and a dynamic EC2 IP required DuckDNS refresh automation.

Accomplishments that we're proud of

  • Completed an end-to-end AWS demo of plan proposal, simulation, human approval, execution, and verification.
  • Split the application into frontend, backend/agent, MCP, ML, and database services using Docker Compose.
  • Moved the AWS deployment to private managed PostgreSQL on Amazon RDS with pgvector.
  • Centralized container logs in CloudWatch and verified an API request-latency metric using real requests.
  • Published a password-protected HTTPS demo and confirmed the site and API were reachable from a phone and a MacBook.
  • Kept the database and internal service ports off the public ingress path.

What we learned

We learned that an agent workflow is more than generating a plausible response: resource constraints, simulation, explicit approval, execution, and verification all matter. MCP creates a useful boundary between an agent and operational tools, but its transport and tool calls still need protocol-level testing. We also learned how easy it is for dependency selection, ignore rules, environment-file handling, IAM identity context, and structured-log filters to affect an otherwise working deployment. CloudWatch alarms showing OK with no datapoints are not proof that all error paths work, so alarm behavior needs deliberate tests.

What's next for NexCord

The next iteration would add per-user authentication and role-based authorization, stronger operational safeguards and failure-injection tests, live camera/video ingestion instead of only sample-image person detection, clearer user-facing execution-state narration, and broader tests of MCP protocol negotiation and tool behavior. We would also run repeatable load tests and trigger controlled failure events to validate error alarms before treating the monitoring setup as operationally complete.

Built With

Share this project:

Updates

Submission history