Inspiration We wanted to build something closer to Jarvis than a regular chatbot, an AI that doesn't just answer you, but actually understands a goal and goes and does it. Most AI tools stop at giving you information or a single response. We wanted MELO to take a goal, plan out the steps, use tools to get it done, and check its own work, the way a real assistant would.

What it does MELO is an autonomous AI agent that takes a natural-language goal from the user and turns it into action. It plans the steps needed, executes tools through n8n workflows, observes the results, evaluates whether the goal was actually achieved, and replans if it wasn't, looping until the task is done.

How we built it MELO runs on a stack of NVIDIA Nemotron models for goal understanding, planning, and decision-making, paired with n8n for tool orchestration and execution. We use multiple AI agents, meta/llama-3.2-11b-vision-instruct, nvidia/nemotron-3-super-120b-a12b, nvidia/nemotron-3.5-lightning-30b-a3b, and nvidia/nemotron-3-ultra-550b-a55b, and the system automatically switches between them if one fails to respond or its server goes down, so MELO stays operational even if a single model provider has issues.

The system is split into four core components built by a 4-person team:

  • MELO Core handles Nemotron integration, planning, and the decision engine
  • n8n/Orchestration manages the tool registry and executes actions through workflows and API integrations
  • Frontend visualizes the goal input, task plan, and activity timeline for the user
  • Observer/Evaluator normalizes tool results, checks task completion, and triggers replanning when needed

A shared schema/contracts layer keeps all four components talking to each other consistently.

Challenges we ran into n8n gave us a fair share of trouble, getting workflows to talk correctly with the rest of the system and debugging why certain triggers weren't firing as expected took a lot of trial and error. We also ran into server issues partway through the build, which meant scrambling to recover progress and get things back up and running under time pressure.

Accomplishments that we're proud of Getting the full loop working end to end, from a user's goal, all the way through planning, execution, observation, and evaluation, was a big win given the time constraints. We're also proud of building a fallback system that switches between multiple AI models automatically, so MELO doesn't just stop working if one model or server fails. Coordinating four independent services into one working system as a team, without everything breaking at the seams, felt like a real accomplishment on its own.

What we learned We learned a lot about how agentic systems actually work under the hood, the gap between an AI "deciding" something and an AI "doing" something is bigger than it looks. We also picked up a lot about orchestration tools like n8n, building resilient systems that can handle model or server failures gracefully, and about working across services as a team when everyone owns a different piece of the pipeline.

What's next for MELO We'd like to expand MELO's tool registry so it can handle a wider range of real-world tasks, make the replanning logic smarter, and give it memory across sessions so it can build on past goals instead of starting fresh every time.

Built With

Share this project:

Updates

Submission history