TaskPilot

Inspiration

Planning a project is easy; turning that plan into action is where things get difficult. We wanted to explore how an AI assistant could go beyond answering questions and actually help people organize and execute their work.

That idea inspired TaskPilot, an AI-powered task planning assistant designed to transform natural-language goals into structured, actionable tasks. For the Alexa+ hackathon, we wanted to explore the potential of agentic workflows and the Model Context Protocol (MCP) to connect AI-generated plans with real tools and stateful actions.

What it does

TaskPilot turns a user's goal into a structured plan with clear, actionable steps. Users can review the proposed plan before approving its execution.

Its core features include:

AI-powered planning: Uses Google's Gemini API to generate structured plans from natural-language requests.
Human-in-the-loop execution: Lets users review and approve a plan before executing task-related tools.
MCP tool integration: Exposes list_tasks, create_task, and complete_task through a self-hosted MCP server using Streamable HTTP.
Shared task state: Keeps task operations through the MCP server synchronized with the application's REST API.
Resilient fallback: Uses bounded retries, a secondary model, and a structured offline planner when Gemini is temporarily unavailable.
Task tracking: Allows users to view tasks and mark them as completed through the application.

How we built it

We built TaskPilot using TypeScript across the frontend and backend, with React and Vite powering the user interface and Node.js with Express handling server-side operations.

Google's Gemini API powers natural-language planning. The backend validates and structures the generated plans, handles transient service errors, and falls back to an offline planner when necessary. The Gemini API key remains on the backend rather than being exposed to the browser.

For the Alexa+ track, we implemented a self-hosted MCP server using the official @modelcontextprotocol/sdk and StreamableHTTPServerTransport. The server exposes three tools: list_tasks, create_task, and complete_task.

We also built an MCP integration test client to verify tool discovery, tool execution, and task-state synchronization. Automated tests cover input validation, error handling, model fallback behavior, and offline planning.

Challenges we ran into

One of our main challenges was handling temporary Gemini API failures. We encountered HTTP 503 responses caused by high demand, as well as an outdated model configuration. We updated the model priority configuration and implemented bounded exponential-backoff retries, secondary-model fallback, and offline planning.

Another challenge was ensuring that the MCP server and REST API operated on the same task state. We addressed this by testing task creation and completion through MCP and verifying that the updated state was visible through the REST API.

We also focused on security and reliability: validating incoming data, keeping API credentials server-side, sanitizing error messages, and ensuring that automated tests could verify failure scenarios without requiring a live API key.

Accomplishments that we're proud of

Built a working AI planning interface with a review-and-approve workflow.
Implemented an MCP server using Streamable HTTP for the Alexa+ track.
Exposed three functional task-management tools and verified their execution through an MCP client.
Verified shared task-state synchronization between MCP and REST endpoints.
Added automated tests for transient API failures, retries, secondary-model fallback, and offline planning.
Successfully generated a Gemini-sourced plan in the browser during manual testing.
Created a resilient architecture that can still provide structured plans when the AI service is unavailable.

What we learned

We learned that building a useful AI agent involves more than connecting a model to a prompt. Reliable tool execution, state synchronization, input validation, error handling, and clear feedback are equally important.

Working with MCP helped us understand how tools can be exposed through a standardized protocol so that AI applications can discover and invoke capabilities. We also learned to distinguish a successful automated test from a live integration test and to design graceful fallback behavior for unreliable external services.

Most importantly, we learned to treat AI as one component of a complete application rather than allowing the entire experience to depend on a single model request.

What's next for TaskPilot

Our next steps are to improve TaskPilot's responsiveness under AI service load, refine its task-execution experience, and expand its tool ecosystem.

We plan to explore persistent task storage, additional integrations, richer task dependencies, and more context-aware planning. We also want to investigate connecting TaskPilot to an actual Alexa+ experience so that users can move from natural-language requests to approved actions through a more conversational interface.

Our long-term vision is to make TaskPilot a dependable agentic productivity assistant that doesn't just suggest what to do next, but helps users turn their goals into completed work.