Just-Enough-AI ⚡

Hardware-aware, memory-grounded, cost-tiered universal meta-harness for AI coding agents.

💡 Inspiration: The AI Coding Paradox

Modern AI coding agents burn through frontier model tokens on every keystroke. Developers routinely spend $20–$100+ a month calling massive 200B+ frontier models for simple tasks like writing a 1-line docstring, fixing a syntax typo, or adding type annotations. Meanwhile, their local machines sit with idling NVIDIA GPUs and pre-installed developer tools with generous free tiers.

We asked a simple question: Why hire a neurosurgeon to apply a band-aid?

We built Just-Enough-AI to democratize agentic coding: a hardware-aware universal meta-harness that automatically discovers and prioritizes free local compute (Ollama) and installed agent CLIs (Google Antigravity, Claude Code) at $0.00, cascading to NVIDIA Nemotron models on Nebius Token Factory only when high-order architectural reasoning is truly needed.

⚡ What It Does

1-Command Zero-Cost Auto-Detection (uv run jeai setup): Instantly scans the host system for local Ollama weights and headless subshells for Claude Code and Google Antigravity, configuring them as Tier-0 and Tier-1 free compute backends.
Dynamic Multi-Provider Waterfall Cascade: If a local model runs out of VRAM or an agent hits a 429 quota exhaustion, Just-Enough-AI automatically cascades to the next best provider and escalates to NVIDIA Nemotron on Nebius Token Factory in under 350ms without interrupting the developer's session.
Speculative Agentic Drafting: Dispatches Nemotron Nano for an ultra-fast ~300ms draft ($0.0001). If deterministic checks pass, it merges with 92% cost savings; if it fails, it bundles the failure trace into a Contrastive Warm-Start Artifact and escalates to Nemotron Super or Ultra.
Differential AST Scope Lock: Prevents small models from causing collateral damage. All code outside the assigned task scope is parsed into an Abstract Syntax Tree, cryptographically locked, and immutably restored before code ever touches disk.
Laptop-OFF Mobile Web Cockpit: A responsive web application with Server-Sent Events (SSE) streaming, live provider status badges, syntax-highlighted diffs, and one-tap GitHub commits—allowing developers to review and approve PRs from their smartphone.

🛠️ How We Built It

Just-Enough-AI is engineered in Python with FastAPI, Rich, Typer, and pynvml, centered around 5 technical breakthroughs:

Hardware-Headroom KV-Cache Governor: Directly queries GPU driver telemetry via pynvml to calculate real-time physical attention memory footprint: $$\text{Memory}{\text{KV}} = 2 \cdot B \cdot L \cdot H \cdot D \cdot \text{precision_bytes} \cdot \text{tokens}$$ Dynamically calculates available headroom $H{KV}$, triggers semantic AST context pruning, and prevents CUDA Out-Of-Memory crashes before allocating context.
In-Harness HelpSteer2 Alignment PR Gate: Implements NVIDIA's HelpSteer2 multi-attribute rubric as a runtime pre-merge gate: $$\text{Gate Score} = 0.40 \cdot \mathcal{C} + 0.30 \cdot (1 - \mathcal{R}) + 0.20 \cdot \mathcal{S} + 0.10 \cdot \mathcal{V}$$ Evaluating Correctness ($\mathcal{C}$), Regression Risk ($\mathcal{R}$), Security ($\mathcal{S}$), and Conciseness ($\mathcal{V}$).
Polymorphic Provider Engine: Modular ProviderAdapter interface supporting direct OpenAI-compatible HTTPS streaming (NebiusDirectAdapter), local HTTP daemon (OllamaLocalAdapter), and sandboxed PTY subprocesses (AntigravityCliAdapter, ClaudeCodePtyAdapter).
Compiler-Grade AST Security & Fuzzing: Subprocess sandbox with path traversal defense, ANSI sequence sanitization, credential masking, and automated boundary fuzzing across adversarial inputs.

🧗 Challenges We Faced

Eliminating Collateral Code Damage: Open-weight models often solve the requested function but accidentally reformat or truncate surrounding functions. Designing the Differential AST Scope Lock required comparing AST symbol trees and cleanly re-inserting original immutable nodes without corrupting comments or whitespace.
Cross-Platform CLI Process Sandboxing: Capturing streaming output from interactive terminal agent tools without hanging on ANSI terminal control codes required a custom PTY buffer stripper.
Physical Memory Accounting: Accurately budgeting dynamic KV-cache memory across heterogeneous architectures (MQA vs. GQA) required deep calibration with NVIDIA Nemotron model specifications.

🏆 Accomplishments We're Proud Of

95.9% Dollar Savings: Evaluated across our 50-task benchmark suite, dynamic tier routing slashed token expenditure from $0.0425 (GPT-4o) down to $0.0017, while maintaining a 100% verification pass rate.
100% Codebase Scope Integrity: 0 unintended symbol mutations across all evaluated benchmark runs thanks to the AST Scope Lock.
Production-Grade Reliability: 99 passing unit tests covering providers, security sandbox, AST lock, router, KV governor, and SSE server.
Zero-Downtime Cascading: Successfully demonstrated hot failover from quota-exhausted free CLI tools to Nebius Cloud Nemotron in <350ms.

🧠 What We Learned

Small, focused models paired with strict compiler-grade AST gates and alignment rubrics (NVIDIA HelpSteer2) frequently match or beat unguided 200B+ models on scoped engineering tasks at a tiny fraction of the cost.
Nebius Token Factory's exceptional Time-to-First-Token (~300ms on Nemotron Nano) is the key enabler that makes speculative agentic drafting viable at the CLI level.

🔮 What's Next for Just-Enough-AI

Server-Side KV-Cache State Sharing: Extending Nebius Token Factory with prefix-caching endpoints for cross-task prompt reuse.
IDE Extensions: Native plugins for VS Code and Cursor to show live routing tiers and token cost counters in the editor status bar.
Multi-Agent Consensus: Speculative multi-draft voting across diverse local models before cloud escalation.