Pineagent

GIF

Inspiration

AI coding agents are great when you work alone, and chaotic when a team works on the same repo. Two people (or two agents) edit the same file, someone commits a teammate's half-finished work, and nobody can tell what the agent is doing until it's done. We wanted a shared workspace where people and agents edit one repository together with clear ownership — and where you can see the agent think, on a little robot plant sitting on your desk.

What it does

Pineagent is a collaborative, browser-based workspace for a GitHub repository, plus a physical plant companion that shows what's happening.

Sign in with GitHub, open a repo you can push to. The server clones it with your GitHub token. The browser never sees a GitHub token.
Edit together, live. Everyone in a branch sees each other's edits as they type, who has which file open, and who is editing what.
One writer per file. Your first keystroke takes the file's lock; everyone else gets a read-only view with "Alice is editing". Locks survive Wi-Fi blips and server restarts.
Commit only your own work. The server tracks who edited each file. You can commit and push your changes, never a teammate's uncommitted work. Push is one job: commit → pull --rebase → push, and it stops cleanly on conflicts.
A workspace that feels like an IDE: file explorer with create file/folder, Git status letters, split editors, and profile cards that jump you to what a teammate is working on.
An agent run panel that shows each run's plan, routing decision, approvals and progress. (Runs use a simulated agent in our local demo while we finish wiring the real harness into the live server.)
The plant robot (ESP32-S3) plugs into your laptop over USB and connects straight from the browser with one click (Web Serial). It mirrors your agent on an animated face — planning, routing, executing, waiting for approval, done, error — and talks: "Hmm, let me think.", "Psst! Can you pick one for me?", "Yay! All done!". It waves its arms when a run finishes, chirps its piezo when it needs your choice, and greets teammates who join. Unplug it and it picks up where it left off.

How we built it

Frontend: React + TypeScript + Vite, Monaco editor, Socket.IO client. Edits are sent as small operations against a versioned file; out-of-date edits are rejected and the client resyncs.
Realtime server: Node.js + Socket.IO. One process is the authority for locks, file versions and a per-branch Git queue (one Git job at a time per worktree).
Auth: Auth0 with GitHub social login. The server uses an Auth0 machine-to-machine app with Private Key JWT (no client secret) to fetch each user's GitHub token from Auth0, checks push access with GitHub's API, and hands the token to Git only through GIT_ASKPASS.
Agent harness: a TypeScript runtime that generates candidate plans as a dependency-aware DAG, routes between them with an OpenAI-based decision model (with a backup scorer), synthesizes a patch once, verifies it, and can fall back to a warm coding agent. Routing failover and coding recovery are reported separately.
Cloud (AWS, Terraform): one EC2 instance behind Caddy (HTTPS via Let's Encrypt) on an Elastic IP; repos on an encrypted EBS volume that survives instance rebuilds; DynamoDB for lock/version/editor restart recovery; deploys go to S3 and are installed over SSM with a health check and automatic rollback. No SSH, IMDSv2 only, least-privilege IAM.
Hardware: ESP32-S3, a color display, two servos, I²S amplifier and speaker, and a piezo. 18 face animations designed in Rive, pre-rendered, palette-encoded and compressed into flash. A line-based text protocol over native USB (STATE planning, SERVO WAVE, PIEZO CHIRP), and a safety gate that refuses servo and speaker commands unless external 5 V is present while the display keeps working on USB power. Without 5 V, the browser falls back to the piezo and the laptop speaker.
Voice: each line was recorded once with ElevenLabs (we also tried Amazon Polly), trimmed and stored with the web app. The browser streams it to the robot over USB in 512-byte acknowledged chunks into PSRAM, and the robot plays it on its speaker.
Testing: Vitest unit tests and Playwright end-to-end tests with real browsers against a mock server (two users fighting over a lock, reconnects, commit ownership).

Challenges we ran into

Realtime correctness: two people typing in the same file within milliseconds must end with exactly one writer and no lost text. We used single-writer locks with leases and heartbeats instead of merging edits, and versioned every change.
Not leaking credentials: GitHub tokens never touch the browser, the repo, .git/config or logs.
Restarts mid-edit: a deploy used to drop every lock. We write lock and version changes through to DynamoDB before acknowledging them, and restore them on startup.
Hardware power safety: the servos and amplifier need a separate 5 V rail that must never back-feed the ESP32.
Getting audio to the robot: our first version streamed a whole clip at once and only 20% arrived — the chip's USB receive buffer silently drops what doesn't fit. Sending it in small chunks the robot acknowledges fixed it (76 KB in under a second).

Accomplishments that we're proud of

It's live at https://pineagent.us, with real GitHub sign-in, real clones, and commits that land on GitHub authored by the right person.
Automatic rollback: a broken release rolls itself back.
A plant robot with 18 expressive animated faces that talks, waves and chirps along with your agent, driven straight from the browser over Web Serial.

What we learned

WebSockets and Socket.IO from first principles, lock leases and versioned edits, Auth0 token handling, Terraform and AWS from scratch, and how much hardware power design matters before you plug anything in.

What's next for Pineagent

Wiring the agent harness into the live server so runs edit the shared repo under the same locks, talking to the plant (voice tasks), per-user sandboxes with a terminal, and pulling remote changes into open branches.

Credits and AI disclosure

Built during SwampHacks XII. We used AI coding assistants (Claude Code) while building, with every change reviewed by a human owner. Open-source libraries and services we built on: React, Vite, Monaco Editor, Socket.IO, jose, AWS SDK, Terraform, Caddy, Auth0, Arduino-ESP32, PlatformIO, Rive, the Web Serial API, ElevenLabs and Amazon Polly (voice), Playwright and Vitest.