Inspiration
Current agentic development suffers from a fundamental design flaw: financial and cognitive collapse caused by an absolute dependence on proprietary cloud infrastructures. Common commercial tools operate under a linear autoregressive paradigm that saturates context windows by repeatedly sending raw source code in plain text. This skyrockets token consumption costs to levels that are unsustainable for independent developers, startups, and industrial-grade systems software.
SODA (Software Orchestration & Development Agency) was born out of the necessity to break this monopoly. It introduces a Local-First hybrid architecture based on the Teacher-Student paradigm and the physical truth of the disk. The goal is straightforward: leverage cloud infrastructure for heavy abstract reasoning while deterministically distilling that knowledge into local Small Language Models (SLMs) that run on consumer hardware at zero inference cost.
What it does
SODA is a distributive agentic execution and compilation environment based on strict hardware and software quality control, operating across two main layers:
- SODA FUSION (Teacher Cloud): Orchestrates complex software development by dynamically balancing the high throughput of Gemini Flash with the deep abstract reasoning of Gemini Pro. It validates hot-swapping code mutations through real Abstract Syntax Tree (AST) parses via
Tree-sitterand SHA256 hashes before persisting data to disk via a transactional linker (Git Sentinel). - SODA BETA (Student Local): Runs silently in the background (Shadow Mode) on the local machine. It captures successful developer interactions using low-latency asynchronous daemon threads to structure local fine-tuning datasets. This enables a 1.5B model (Qwen 2.5 Coder) to learn and operate natively on consumer hardware utilizing less than 6GB of VRAM.
How we built it
The system core was built entirely using Python and structured through a decoupled dependency injection container (kernel/core/service_container.py) to completely eliminate massive, monolithic coupling (God Objects).
Concurrency was engineered on top of asyncio, implementing an OrchestratorKernel pattern featuring granular locks per project (asyncio.Lock) and strict concurrent semaphores to mitigate Google API rate limits. Blocking synchronous calls to the Docker SDK were isolated using asyncio.to_thread to keep the system's main event loop free. The local training pipeline was written utilizing QLoRA in bfloat16 quantization techniques powered by NF4 bitsandbytes and Gradient Checkpointing.
Challenges we ran into
The first major obstacle was the stochastic nature and raw hallucinations of LLMs when generating code. We solved this by transforming the API execution into a closed feedback loop where the local CodeGenerator output is intercepted by a loop-back Evaluator Node (Gemini 3.5) acting as a binary interface judge.
The second major challenge was the initial lack of success traces needed to feed SODA BETA's training process. We bypassed this through automated reverse semantic engineering: taking real production code that successfully compiled in the local environment, extracting its AST, and programming Gemini 3.5 to reconstruct backward the exact hierarchical target tree (GoalTree) and the fuzzy original prompt that generated it.
Accomplishments that we're proud of
We are deeply proud of achieving a strict system for Hardware-Aware Throttling. By natively integrating pynvml to monitor our local GPU (NVIDIA RTX 5070 Ti), the orchestrator automatically degrades the concurrency of tasks dispatched by the Recursive Planning Council if the video memory utilization exceeds 90%:
$$\text{VRAM}{\text{available}} = \text{VRAM}{\text{total}} - \text{VRAM}{\text{consumed}}$$ $$\text{If } \left( \frac{\text{VRAM}{\text{consumed}}}{\text{VRAM}{\text{total}}} \times 100 \right) \ge 90.0\% \implies \text{Parallelism}{\text{limit}} = \text{max_evaluators} - k$$
Making an agentic software stack completely aware of the thermal and memory constraints of the physical machine hosting it is a massive milestone for industrial-grade robustness.
What we learned
We learned that the true intelligence of an AI development ecosystem does not reside in the sheer parameter scale of an isolated model, but rather in the deterministic scaffolding that governs it. Forcing a probabilistic language model to behave like a traditional compiler via rigid syntactic AST validation before executing the code physically inside isolated sandboxes is the only real way to eliminate software hallucination at scale.
What's next for soda
The next critical milestone on our infrastructure roadmap is the implementation of EVCM (Visual Code Compression Mapping). This is a proprietary protocol that transmutes the source code of logical layers and dictionaries into dense, two-dimensional matrices of discrete pixels ($1 \text{ Character} = 1 \text{ Pixel}$).
By applying computer vision algorithms (OpenCV) over these 1 Megapixel ($1024 \times 1024$) canvases, we will compress the highly redundant visual patterns native to programming syntax:
$$f: \text{Source Code} \longrightarrow \text{SODA_Tag}$$ $$f^{-1}: \text{SODA_Tag} \longrightarrow \text{Executable Code}$$
This approach will allow us to execute millisecond-level hot-swapping of 20MB specialty LoRA adapters over a single static base model in VRAM in under 50 milliseconds, locking in absolute cloud independence and driving token costs down to zero.
Log in or sign up for Devpost to join the conversation.