UI App
CLI App

Project Story

About the Project

This project was born from a simple observation: despite the rapid rise of LLM-powered assistants—from Apple Intelligence to OpenAI’s offerings—most solutions feel limited and unsatisfying. They often operate in restricted sandboxes, without real access to the computer environment, making it impossible to build truly useful workflows or complete end-to-end projects.

On the other hand, current coding agents may generate code or run snippets, but they still lack a global, secure access to the operating system, which prevents them from becoming reliable partners for real-world software development.

We wanted to explore what an assistant could look like if it combined:

the power of LLMs,
the discipline of Clean Architecture,
and secure but flexible access to system tools.

That vision became GPT-OSS Hackathon.

Inspiration

We were inspired by the idea of building an open, local agent that could interact with files, applications, and system controls—while respecting strict safety rules. Instead of yet another “chatbot,” we aimed for a tool that feels closer to a real operating system companion, capable of executing commands, managing projects, and helping with code navigation.

This is our answer to the limitations of Apple Intelligence and OpenAI integrations: a system that is local, extensible, and transparent.

How We Built It

We designed the project following Clean Architecture principles:

Entities represent the core objects (files, LLM).
Use Cases capture workflows (file operations, project navigation, text generation).
Ports & Adapters ensure clear separation between abstract logic and concrete implementations (file system, APIs).

The application exposes multiple entry points:

a CLI for demonstrations,
a FastAPI HTTP server with endpoints and a minimal UI,
a desktop UI for native interactions,
and an interactive coder with tool tracing and live confirmation of commands.

We also implemented a Tools API where the LLM can call functions such as files.read, system.set_volume, or git.status, with safety guards like HACK_WORKSPACE_ROOT and explicit user confirmation for sensitive operations.

What We Learned

How to apply Clean Architecture in a practical LLM project, ensuring testability and maintainability.
The importance of fine-grained safety controls, such as requiring confirmation for non-predefined commands.
How to integrate multiple domains (files, system, applications, Git, web) into a coherent toolset.
The value of building UI layers (web and desktop) that make the system accessible to both developers and end users.

Challenges We Faced

Balancing power and safety: giving the LLM enough freedom to be useful, while enforcing strict boundaries to avoid accidental file deletion or unsafe system commands.
Managing complexity: integrating many tools (files, system, Git, applications) while keeping the architecture modular.
Latency and orchestration: ensuring smooth tool invocation flows, especially when chaining multiple steps.
Designing a universal interface: building APIs, CLI, and UI in a way that remains consistent and extensible.

Conclusion

With GPT-OSS Hackathon, we created a local, secure, and extensible agent that showcases what assistants could achieve if they were truly integrated into the operating system.

Instead of a black-box “AI feature” like Apple Intelligence, our project is: