Operon

Operon extends VS Code with durable task execution, streamed workflow updates, and cloud-ready development operations.

Comment

Operon: Elastic-Powered, Safety-First AI Developer Agent for VS Code

Inspiration

Current AI coding tools often suffer from three major issues:

  1. Ambiguous or Vague prompts causing incorrect code modifications.
  2. Accidental codebase corruption due to unrestrained tool execution.
  3. API rate limits or network outages causing complete developer lock-outs.

Furthermore, multi-agent frameworks often struggle with role clarity, letting agents wander outside their designated responsibilities. We wanted to build a system that acts like a disciplined software engineering guild: classifying tasks, scanning dependencies, planning thoroughly, validating changes in an isolated sandbox, and enforcing strict, dynamic boundaries on what each agent role is allowed to do.

What it does

Operon is a complete AI engineering agent integrated directly into VS Code. It runs a deterministic ==10-stage agentic workflow== to safely solve coding tasks from your IDE:

  • [x] Project Mode Detection: Classifies task intent, risk level, and context requirements.
  • [x] Context Collection: Scans repository hierarchy, compiles L0-L4 snapshots, and performs AST parsing.
  • [x] ClarityForge Ambiguity Gate: Pauses execution if the prompt is too vague, proposing assumptions or prompting for clarification.
  • [x] BlueprintSmith Planning: Formulates architecture-aware plans, file targets, and risk outlines.
  • [x] TaskWeaver Todo Decomposition: Slices plans into atomic tasks inside a Todo Ledger.
  • [x] LLM Planner: Produces structured planning outputs and triggers a pause/approval gate for high-risk files.
  • [x] CodeMason Execution Guild: Executes tasks, applying patches using strict write allowlists.
  • [x] Tester Sandbox & Acceptance Validator: Runs sandbox tests to verify correctness and rejects trivial successes.
  • [x] Debugger Loop: If tests fail, runs a Root Cause Analysis (RCA) loop to locate and repair failures (up to 3 iterations).
  • [x] Lifecycle Logging & State Commit: Commits run manifests and audit logs to SQLite/PostgreSQL and Elasticsearch.

How we built it

Operon's full-stack codebase is built around clean architectural boundaries:

  • The Orchestrator: managing the sequential transition between the agent stages.
  • The IDE Extension: Built using TypeScript to interface directly with the FastAPI server via Server-Sent Events (SSE) for live timeline tracking.
  • Dynamic Role Profiling (skill.md + Elastic): We designed 11 agent role profiles under docs/agents/skills/. Using the [skill_profile_loader.py], these are parsed and ingested into Elasticsearch's agent_behavior_index. The [Context Memory Agent] queries this index dynamically to assemble a Behavior Pack that locks active agents inside strict sandboxes.
  • Safety and Verification Layer: for allowlisted write zones/Git rollbacks, and tester_sandbox for safe test execution.
  • The Context Engine: utilizing hybrid lexical and semantic ranking via Reciprocal Rank Fusion (RRF).

Reciprocal Rank Fusion (RRF) Context Retrieval Formula

To rank codebase snippets, we combine BM25 lexical search and dense vector search results using Reciprocal Rank Fusion (RRF)[^1]. The RRF score for a document \(d\) is computed as:

$$ RRF(d \in D) = \sum_{m \in M} \frac{1}{k + r_m(d)} $$

Where:

  • \(M\) represents the set of ranking models (lexical and semantic vector models).
  • \(r_m(d)\) is the rank of document \(d\) under ranking model \(m\).
  • \(k\) is a constant parameter (default is \(60\)) that mitigates the impact of low-ranked outliers.

Challenges we ran into

  • Strict Agent Boundary Control: Without constraints, agents frequently hallucinate capabilities or bypass prompt instructions. Resolving this required building the custom Markdown parser to convert human-readable skill.md profiles into structured Elastic queries.
  • Managing LLM Context Windows: Codebases quickly exceed prompt window limits. We designed a custom ContextPack system to build unified, lightweight representations (L0-L4) of the repo rather than dumping raw code.
  • Zero-Downtime Fallbacks: Coordinating Ollama to start, pull a model like qwen2.5-coder:7b, and resume a task seamlessly after a Gemini 429 / 403 API failure required robust error interception in our LLM router.

Accomplishments that we're proud of

  • Self-Healing Local Fallback: Automatically starting, pulling, and transitioning LLM task orchestration to local Ollama services upon cloud API failure, meaning the developer is never blocked.
  • Elastic-Indexed Governance: Successfully implementing dynamic constraints parsing, where human-readable Markdown profile definitions (skill.md) translate directly to runtime execution boundaries.
  • Unified IDE Streaming & Verification: Building a smooth, stream-first VS Code extension interface using Server-Sent Events that provides real-time progress while executing robust sandbox tests and automated debugger code corrections.
  • Automated Debugging Limits: Safe and bounded self-correction in the sandbox, ensuring debug iterations strictly satisfy: $$ \text{Iterations} \le 3 $$

What we learned

We learned that AI agents perform drastically better when they plan before coding. Governing agent interactions using structured input/output contracts and strict allowed/forbidden tool boundaries (loaded dynamically from Elastic) drastically reduces hallucinations and prevents system corruption.

What's next for Operon

  • Full Specialist Process Isolation: Decomposing all remaining in-process agents (frontend, backend, cloud-devops, security, etc.) into standalone, containerized micro-services.
  • Native Elastic Vector RAG: Transitioning the context ranker to a native Elasticsearch hybrid dense-sparse vector pipeline instead of candidate-local rank fusion.
  • Shared Deployments with PostgreSQL: Production worker scale orchestration running tasks off PostgreSQL-backed queues with robust execution lease management.
  • Advanced Compliance & Security: Integrating signed build provenance/attestations, generic high-entropy secret scrubbers, and stricter enterprise governance policies.

What Makes Operon Different (USP)

USP Feature Description Support Tech
Elastic-Governed Behavior Roles defined in editable markdown (skill.md), compiled dynamically as behavior constraints. Elasticsearch, skill_profile_loader.py
Cloud-to-Local Hybrid Router Automatically fails over to local Ollama (qwen2.5-coder:7b) if cloud APIs throw rate/auth errors. gateway.py, Ollama daemon auto-start
Deterministic Sandboxing Runs patches in isolated environment. Auto-triggers a 3-pass RCA Debugger on test failure. Pytest sandbox, stacktrace parser
Git-Backed Safety Gate Restricts edits to workspace paths. Automatically rolls back files if validation tests fail. Git checkout/restore hooks, CodeMason

[^1]: Reciprocal Rank Fusion (RRF) is a state-of-the-art technique for merging multiple ranked lists into a single consolidated result list without requiring score normalization across different index models.

Share this project:

Updates