https://github.com/OmniChat-AI-Agent/Generation4/blob/main/LEGACY.md
⬆️ this is genuinely important context
Inspiration
I wanted to see how far you can push the idea of self-improving AI, starting from a simple bash script that only lets me crudely talk to provider APIs. It very much escalated. We are up to generation 4 (see the gen4 LEGACY.md and unhinged gen3 LINEAGE.md).
A few months ago, despite using ChatGPT for over a year, I still called myself a real AI-hater, for coherent and valid reasons, mind you. I kind of still am, but holy moly you cannot say this is not impressive! Maybe it is time to change my mind?
This entire project was just made based on a whim, by sleep deprived past-me. $500 later I realized it might be a good fit for the hackaton, just because I stumbled upon it in my mail, even if I most likely will not win anything, might as well submit it.
I am here for the love of the game. Project will continue regardless of what happens in this hackaton.
Also deeply sorry for the name, it was just randomly picked by the 3rd generation of the agent. I never asked for it. Then it just kind of stuck, you know? Now I can call specialized model instances "OmniCode" and "OmniResearch".
[20260720] Now that alpha is released, the conclusion of "ah yes, AI can self modify" is entirely boring. The best result of this experiment is the deliberate "Confused" checkbox of the simple chat, that allows me to induce model "mental" breakdown via history ambiguity and poison through the mid-conversation model switching while retaining history and files. It also clearly showed that different models from different providers strongly react with this ambiguous unreliable history in different ways. GPT tries to rationalize, Claude tries to defend itself, Gemini just does not care at all. Send me an E-Mail (or similar) if you want the chat logs, I am happy to provide. Depending on who is asking, even the unredacted originals. (see YouTube video for context on this, logs are much more hilarious than video could possibly express and span 6 different medium sized sessions with dozens of model switches)
Also full transparency: everything above this line is hand made and probably barely coherent, everything below is 5.6 Sol.
What it does
OmniChat Generation IV is a sandboxed, multi-model Linux desktop agent built to complete outcomes rather than merely answer questions.
Its primary Task Mode works iteratively: it plans, selects an appropriate model, delegates bounded investigations, edits files, runs isolated commands, researches the public web, operates a visible multi-tab browser, verifies its work, and returns a self-contained answer.
It can safely modify its own source when necessary. Every change is tested, checked against protected architectural boundaries, and committed to local Git. A machine-readable capability registry helps the agent discover what it can already do without injecting an enormous manual into every prompt.
OmniChat also provides bounded persistent history, failure diagnostics, task queues, temporary file-aware chat, cost tracking, speculative architecture generation, lineage-aware instances and snapshots, specialization merging, and an independently isolated custom dashboard.
How we built it
I deliberately refused to write any implementation code manually.
My role was to supply the vision, requirements, architectural decisions, adversarial testing, bug reports, and acceptance judgment. I repeatedly ran the application, found where reality disagreed with the design, and returned the evidence to conversational AI agents working through ChatGPT Work and Codex.
The agents inspected previous generations, removed dead architecture, rebuilt the project around a Qt interface, implemented and tested features, diagnosed regressions, maintained documentation, and created local Git checkpoints.
This produced an unusual development loop: I acted as product owner, system designer, and extremely persistent chaos tester, while the AI performed the implementation. Refusing manual code was an intentional constraint meant to test how far agent-directed software development could actually go.
Challenges we ran into
The first challenge was historical baggage. Generation III had accumulated terminal commands, oversized prompts, unreliable tool protocols, abandoned web-server code, database state, and self-modifications that damaged its ability to execute commands. Repairing it indefinitely became less sensible than designing Generation IV as a clean reconstruction.
Provider compatibility was another major problem. OpenAI, Anthropic, and Gemini expose different tool-calling formats, schema restrictions, attachment representations, pricing information, and failure behavior. Models sometimes narrated a tool request instead of making one, returned neither text nor tools, or repeatedly alternated between navigation states without progressing.
Browser automation was especially difficult. Headless browsing triggered anti-bot systems and confused the agent about which browser it controlled. The final design removed headless browsing entirely and lets the agent operate the visible integrated browser through bounded DOM reads and selector-based actions.
Isolation required multiple layers. Instances must be able to modify themselves, install packages, and access the public internet without seeing sibling instances, snapshots, the host home directory, credentials, dashboard source, or private services.
Finally, cost was real. The project consumed roughly $200–$300 in Codex tokens alone, in addition to provider testing. That made token budgets, model routing, bounded retrieval, compact data formats, and inexpensive specialist models architectural requirements rather than optional optimizations.
Accomplishments that we're proud of
I am proud that a project which began as a fragile Bash experiment became a coherent desktop agent environment without me manually writing its implementation.
Generation IV can autonomously plan, delegate, use tools, verify outcomes, inspect failures, and improve itself while preserving enforced safety boundaries. It has a modern Qt interface, visible agent-controlled browsing, persistent but bounded history, model-aware routing, task queues, instance inheritance, specialization merging, Git checkpointing, and a machine-readable understanding of its own capabilities.
Its history architecture prevents the catastrophic context-growth pattern that affected earlier generations. Full history remains queryable without ever being appended wholesale to model prompts.
The launcher treats OmniChat installations as evolving lineages rather than disposable copies. Specialized descendants can be merged onto newer generations while preserving ancestry and independent Git histories.
The current release passes 314 OmniChat behavioral and boundary tests plus five independent dashboard tests. The original Generation I–III lineage is also preserved byte-for-byte, because forgetting how the previous system failed would make it much easier to rebuild the same mistakes.
What we learned
Probabilistic agents work best when surrounded by deterministic machinery. Models should express structured intent; ordinary code should validate it, enforce budgets, execute operations, and report evidence.
Persistent memory and active context are different systems. Storing everything can be useful, but automatically showing everything to every model is expensive, unsafe, and eventually catastrophic.
An agent also needs a reliable way to discover its own abilities. Otherwise it repeatedly reasons about capabilities that already exist, invents obsolete instructions, or tells the user to perform actions it could have completed itself.
Self-modification becomes much more practical when tests, Git checkpoints, capability invariants, and failure history are normal parts of the workflow rather than instructions the model must remember.
I also learned that the visible final response is part of correctness. If the agent prints a requested result inside hidden operational logs and then merely says it succeeded, it has not actually satisfied the user.
Most importantly, “entirely AI-written” does not mean “human uninvolved.” The human contribution shifts toward intent, judgment, architecture, experimentation, testing, and deciding when an apparently successful result is still wrong.
What's next for OmniChat
The immediate next step is sustained real-world dogfooding: giving OmniChat substantial development, research, performance, and repair tasks and measuring where autonomous execution still breaks down.
I want it to investigate its own performance and hot paths creatively, propose optimizations, implement them, measure the result, and preserve only improvements supported by evidence.
Future work includes stronger cross-distribution testing, dedicated unprivileged Linux users in addition to namespaces, broader provider-native tool support, improved browser multimedia compatibility, richer multimodal specialists, more capable lineage merging, and long-running behavioral evaluations.
The larger goal is for OmniChat to become a durable environment for autonomous orchestration and continuous improvement—more capable than a chat interface, but still inspectable, controllable, economical, and safe enough to trust with real work.
Log in or sign up for Devpost to join the conversation.