Inspiration
Every AI agent today is stateless — it runs commands for you, but it never learns from how you use it. You might check logs, then check disk space, then list files every single morning, and your agent treats each request like it's the first time. We kept catching ourselves repeating the same sequences of commands and thought: why can't the agent notice this on its own? That's the core idea behind Orbit — an agent that watches itself being used and gets smarter over time, without any manual configuration.
What it does
Orbit is an intelligence layer that sits on top of an AI agent (Hermes) and closes a self-improvement loop. It silently captures every tool call the agent makes, groups them into workflows, and detects repeated patterns using structural matching combined with semantic embeddings. When it finds a pattern you keep repeating, it sends it to NVIDIA's Nemotron-3-Super-120B, which analyzes your specific behavior and generates a ready-to-install automation — complete with a title, reasoning, and working Python code. When you accept the suggestion, Orbit writes a native SKILL.md file that Hermes discovers and uses immediately. The full loop is: observe → detect → suggest → install → use. No configuration, no prompt engineering — the agent learns your habits on its own.
How we built it
Orbit is an intelligence layer that sits on top of an AI agent (Hermes) and closes a self-improvement loop. It silently captures every tool call the agent makes, groups them into workflows, and detects repeated patterns using structural matching combined with semantic embeddings. When it finds a pattern you keep repeating, it sends it to NVIDIA's Nemotron-3-Super-120B, which analyzes your specific behavior and generates a ready-to-install automation — complete with a title, reasoning, and working Python code. When you accept the suggestion, Orbit writes a native SKILL.md file that Hermes discovers and uses immediately. The full loop is: observe → detect → suggest → install → use. No configuration, no prompt engineering — the agent learns your habits on its own.
Challenges we ran into
The biggest challenge was discovering that Hermes's documented APIs for registering skills (ctx.register_skill() and ctx.register_command()) are completely non-functional — both are silently ignored. We spent significant time trying to make them work before discovering that Hermes actually loads skills from SKILL.md files in a specific directory. This turned out to be a breakthrough — once we wrote real SKILL.md files, the install became genuinely real rather than just a database entry.
Nemotron as a reasoning model also had quirks. It sometimes returns output in the reasoning_content field instead of content, and truncates output if max_tokens is below 2000. We had to add fallback handling for both cases.
Another challenge was that the event collector hooks stopped recording new events in certain sessions, which limited us to the patterns detected from earlier sessions. We couldn't create new, more complex patterns during the demo preparation, so we had to work with the existing terminal-to-terminal pattern — which fortunately demonstrated the full loop clearly.
Accomplishments that we're proud of
The moment that made the project feel real was when we ran Use the orbit_run_two_terminal_commands skill to run "echo hello" and then "echo world" — and Hermes actually found the SKILL.md that Orbit had written, understood the skill, and executed both commands. The full loop closed end to end: observe → detect → suggest → install → use. The generated code wasn't a template — Nemotron analyzed our specific behavior and wrote a function that calls the terminal tool twice with the right arguments. We're also proud that the entire plugin is pure stdlib Python with no dependencies, includes safety validation on generated code (blocking path traversal, dangerous imports, and shell injection), and runs inside a sandboxed container.
What we learned
We learned that the gap between "AI generates code" and "AI generates code that actually runs inside a live system" is enormous. Getting Nemotron to produce syntactically correct Python was the easy part. Making that code work as a real, discoverable skill inside Hermes required understanding how the agent runtime actually loads skills (undocumented SKILL.md files, not the documented API), validating generated code for safety before execution, and handling all the edge cases of a reasoning model's output format. We also learned that semantic embeddings (BGE) are essential for pattern detection — purely structural matching would miss variations of the same workflow.
What's next for Orbit
Three directions: goal-level pattern detection — recognizing higher-level goals like "debug a Python error" regardless of which specific tools were used, using Nemotron to reason about intent rather than just matching tool sequences. Cross-session and cross-user learning — sharing detected patterns across users so the agent benefits from collective behavior. And a confidence threshold for auto-install — when Orbit is highly confident about a pattern and the generated skill passes all safety checks, it installs automatically without even asking, making the agent truly self-improving in the background.
Log in or sign up for Devpost to join the conversation.