TryRepo

Every hackathon ends with a folder of GitHub links nobody runs. Paste one into TryRepo and it's live in a minute — a preview URL if it's a web app, a real terminal if it isn't.

Comment

Inspiration

You are about to score this project without running it.

That's not a criticism — judging twenty submissions means reading twenty READMEs, not cloning twenty repos. Nobody clones twenty repos. Running someone else's half-finished project means matching runtime versions, guessing at env vars, and executing a stranger's code on your own laptop.

So the actual work — the repo — goes unopened. That's the sharpest version of a problem covering most of GitHub: the gap between "this looks interesting" and "I have it running" is where open source discovery dies.

We wanted the shortest path from a link to actually using the thing.

What it does

Paste a GitHub URL into a chat. In about a minute you get:

  • Web apps → a live public preview URL. Uses the repo's Dockerfile, or writes one if there isn't one.
  • CLI tools, TUIs, libraries → an interactive browser terminal with the project already built and installed. Not a folder of source — croc --version works the moment the shell appears.
  • Either way → ask questions about the code. The agent greps and reads real files, and shows you which ones it opened.

Everything runs in a sandbox that deletes itself after 30 minutes. Nothing touches your machine.

The obvious approach — a "deploy to cloud" button — fails for a reason we measured rather than assumed. Against a live snapshot of trending GitHub repos, only 4 of 20 shipped a Dockerfile, and 12 of 20 weren't web apps at all. For hackathon repos it's worse. A tool that only handles web-apps-with-Dockerfiles handles a small minority of what people actually want to try.

So tryrepo branches instead of running one fixed pipeline.

How we built it

Daytona — the execution layer, used two ways. It builds an image from an arbitrary repo's Dockerfile and exposes an auto-expiring public preview URL; and it provides PTY sessions that drive a real in-browser shell. The non-web-app branch exists because Daytona exposes PTYs.

CopilotKit — the agent runtime plus interaction patterns that carry the product rather than decorate it. useHumanInTheLoop: when a repo needs API keys, the agent genuinely suspends mid-run, renders a form inline, and resumes with your values — this is what makes config-requiring repos work at all. useFrontendTool: the terminal is a tool whose rendering is a live xterm session, so the agent and user share one surface. useRenderTool: shows the actual files the agent opened while answering questions about the code.

Fireworks AI (minimax-m3) — the judgment that makes the branch possible. One schema-constrained call returns the web-servability verdict, a synthesized Dockerfile when the repo lacks one, the grounded env-var list, and the build steps used to install the project into the terminal image.

Braintrust — reliability as a number. Deploys and terminals are traced in production, and an offline eval scores the model's judgment against 20 hand-labelled real repos using deterministic scorers rather than an LLM judge.

Challenges we ran into

Almost all of these were undocumented behaviour, found by controlled tests against the live API.

Daytona doesn't run your Dockerfile's CMD. The image gives you a filesystem; the entrypoint becomes sleep infinity. The app has to be started explicitly afterward.

A non-root USER line reproducibly breaks sandbox startup — reported as a misleading "failed to resolve container IP," every time. Confirmed by A/B test: identical fixture, USER line the only difference, 2/2 fresh sandboxes failed with it and passed cleanly without. Dropping to a non-root user is a security best practice, so this silently breaks the good repos. We strip it before building.

We shipped a security hole, then fixed it. Terminal session IDs were sequential with no auth on the input/stream endpoints — guessing an ID attached you to someone else's root shell. We'd deferred it as "localhost only," which was wrong: next dev binds 0.0.0.0, so it was live on the venue wifi. Now CSPRNG IDs plus an httpOnly owner cookie, enforced inside getSession() so a new route can't forget the check. Verified: no cookie → 404, owner → 200.

Accomplishments that we're proud of

We measured instead of claiming. Across 20 real trending repos, 16/20 (80%) became something you can actually try — 2 live previews, 14 working terminals. The terminal check isn't "did it start": it runs a command and confirms output comes back.

The eval caught a real hallucination. Checking whether suggested env var names actually appear in the repo caught the model inventing llm_provider_api_keys — a category, not a variable any program reads. Comparing prompt versions:

Scorer v1 v2 Change
env_vars_grounded 89.02% 99.64% +10.63pp — 4 improvements, 0 regressions
web_servable_correct 82.35% 85.29% +2.94pp — 2 improvements, 1 regression

That regression is reported on purpose — it's real instability we'd otherwise be hiding. Three genuinely contestable repos are labelled ambiguous and excluded from scoring; an honest abstention beats a suspicious 100%.

What we learned

Most agent work is discovering how infrastructure actually behaves. Several bugs produced errors pointing at the wrong thing — one PTY failure blamed a missing shell when the real cause was a missing working directory. The only reliable technique was A/B tests against the live API.

Deterministic scorers beat LLM judges when you have ground truth. Our most valuable scorer is a substring check: does this variable name appear anywhere in the repo? Catches hallucination precisely, instantly, for free.

Being precise about what the eval did: we spotted the hallucination by watching the UI, fixed the prompt, then built the eval. Braintrust didn't discover that bug — it quantified the fix at +10.63pp and now guards against regression.

Known limitations (stated on purpose)

  • Synthesized Dockerfiles are best-effort — the model can pick the wrong package manager, same as a human guessing at an unfamiliar repo.
  • HTTP-only: a repo can build and start perfectly and still have no meaningful preview URL if it speaks raw TCP (confirmed with schollz/croc).
  • Workspaces and PTY handles live in memory, so it needs one long-lived process today. The sandboxes are already remote, so that's a shared-volume problem, not an architectural one.

We used Daytona, CopilotKit, Fireworks AI, and Braintrust. We evaluated WorkOS, CodeRabbit, and ElevenLabs and chose not to force them in — a bolted-on integration would have cost us the terminal and the eval, which is where the real work went.

Built With

Share this project:

Updates

Submission history