JARVIS

Mobile phone call with Jarvis

Inspiration

We wanted to do a an assistant based on J.A.R.V.I.S from Iron Man where the user just by talking can change thing surrounding him. In our case we approached it as an assistant that lets you start coding wherever you are as long as you have a phone with you. To add some rigurosity we needed to add a method for the user to check easily wether the code created satisfies him or not.

What it does

JARVIS is a voice-first AI developer assistant that understands your repository and helps execute development tasks end to end. It can route commands, inspect project context, manage memory across sessions, and assist with coding workflows like searching code, preparing patches, and validating changes. It runs through a real-time backend and works across desktop and web interfaces for a consistent experience. It also counts with an IntelliJ plugin to help the developer check wether the code JARVIS made is correct easy and fast with summaries, line by line explanations of changes done and architecture diagrams.

How we built it

We built JARVIS with a modular backend centered on FastAPI, WebSocket-based real-time communication, and agent-driven orchestration. The system uses specialized agents and graph-based task flows for repository reasoning and execution planning, plus memory services for context continuity. On the frontend side, we built React + TypeScript clients, including a desktop app using Tauri and a web app using Vite. We also added safety-focused execution layers, structured event models, and test coverage across API, voice, orchestration, and memory components. We built the plugin with the IntelliJ Platform SDK (Kotlin) as a lightweight bridge between the IDE and JARVIS. It captures editor and repo context, sends it to the backend through API/WebSocket flows, and returns actions/responses directly in the IDE UI.

Challenges we ran into

We saw forwarding voice messages into an AI assistant that then decides where to apply your code wasn't an easy task. We also found a lot of difficulties linking the phone into the computer so the user can code wherever he wants.

Accomplishments that we're proud of

We are really proud to have a real AI voice assistant with whom the user can have a natural conversation and discussion to write his code from anywhere hands-free not just from his computer.

What we learned

Building JARVIS taught us that strong AI UX comes from system design, not just model quality. Real-time voice interactions need low latency, interruption-safe flows, and clear feedback. We also learned that repo-aware automation must be paired with strict safety boundaries, traceable actions

What's next for JARVIS

For us JARVIS is just the start to a future where developers can do their jobs easily and efficiently just by talking, saving them a lot of time and improving their work conditions and mental health.