Slingshot

• 

  Logo

Inspiration

We wanted to build an agent that does something AI still can't do well: optimize code and prove it got faster. An LLM rewrites a function in seconds, it passes review, it ships, and weeks later it turns out to be the slow path nobody noticed. Writing code is easy now but the harder question of "is this actually fast, and is it safe to change?" still falls on a human reading code by hand.

The reason agents are bad at that question is structural. Every new chat starts from zero, so the agent burns its first stretch just re-reading the repo to work out where things live and what connects to what. You can't safely speed up a function without knowing how often it's used, what depends on it, and what breaks if you touch it. An agent that rediscovers the codebase from scratch every session can never hold that picture.

So we built Slingshot to close that gap. It uses GitLab Orbit's knowledge graph to skip the re-reading entirely. The agent queries a live map of the codebase instead of crawling files, and spends that saved effort where it matters. It measures real performance, maps the blast radius, generates fixes, and proves each one with a real benchmark. Slingshot is the optimization loop we always wished we had.

What it does

Slingshot is a GitLab Duo agent that optimizes a function the way a careful engineer would. It runs as a real workflow from start to finish, not a chat that hands you advice.

Point it at a function and it runs a four-step loop:

1. Profile. It triggers a real CI benchmark and captures a runtime and memory baseline. If the project has no benchmark job yet, it writes one from a template and commits it first. Every number comes from a job log, so nothing is estimated.
2. Understand. It queries Orbit's import graph for how widely the function is used, its blast radius, and how far it spreads across projects. Then it classifies the bottleneck as algorithmic, I/O, memory, or concurrency, and scores its impact.
3. Fix. It pulls the minimal Orbit subgraph as grounded context and generates three signature-preserving variants named Speed-First, Memory-First, and Balanced.
4. Prove. It re-runs the same benchmark on each variant and gates on safety. The CI test job must pass and no importer may break, so a fast-but-unsafe variant is disqualified rather than ranked. It then ranks the survivors by measured improvement and opens a merge request once you approve.

It works two ways. Conversationally, you can call any step in Duo Chat, like "profile this", "why is it slow?", "generate fixes", or "validate and rank". Or mention/assign the Slingshot flow agent in an issue with a goal and it runs the whole loop end-to-end. You get back a scorecard where every cell is a real measured number, a confirmed-safe blast radius, and an MR ready to merge.

How we built it

Slingshot is built on top of the GitLab Duo Agent, defined in code so a judge can read every moving part.

• Four skills. Each one is a plain SKILL.md file with YAML front matter and Markdown instructions, no handler code. Every skill is independently runnable, so each can be tested and reused on its own.
• One flow. The file .gitlab/duo-agents/slingshot.yml chains the four skills and passes each step's structured output forward to the next. The agent's system prompt and routing live in AGENTS.md.
• Published to the AI Catalog. We author everything in code, then publish the agent and the Slingshot flow to the AI Catalog in the Duo web UI, where it gets version history and anyone can install it. Orbit is enabled for the agent in the same web UI, so the Knowledge Graph is live the moment the flow runs.
• Orbit for context. The agent reaches Orbit through Duo's native commands get_query_dsl, get_graph_schema, and query_graph. It never crawls files. Instead it asks the graph the questions it would otherwise reconstruct by reading, like which files import a function and how far it spreads across projects.
• No hard-coded queries. Orbit is an official experiment, so its schema and query language can change. The agent fetches the live grammar and schema at the start of each session and composes queries from intent, so Slingshot keeps working as Orbit evolves.
• GitLab CI/CD for the numbers. A generated pipeline carries a benchmark-<function> job and a test-<function> safety job, driven by a Python runner that uses time.perf_counter and tracemalloc. The benchmark produces the runtime and memory, and the test job is the safety gate.
• A demo target project. A process_payment function with a hidden O(n²) fee lookup, imported across billing and API modules, with a pytest suite that acts as the live safety gate.

Three principles shaped every skill. Descriptions say when to trigger a skill rather than what it does, because that is how Duo decides whether to load it. Outputs are positive contracts with exact fields and columns rather than lists of prohibitions. And each skill has a fallback path so it works standalone and reuses context when chained.

Challenges we ran into

• Orbit thinks in imports, not calls. Our whole blast-radius idea assumed a call graph with a :CALLS edge between functions. Orbit has no such edge. It tracks how code connects through the import graph, where an ImportedSymbol resolves to a Definition through an IMPORTS edge. We reframed "call frequency" as import frequency and rebuilt every query around the import graph.
• The graph has its own vocabulary. In Orbit a function is a Definition, a service is a Project, and an imported name is an ImportedSymbol with no name field, only an identifier_name. Composing a query against a node type whose schema we had not confirmed produced silent failures like 'name' is not one of the allowed properties. We made it a hard rule to fetch the schema for each node type before querying it.
• Query live, never from memory. Because Orbit is an experiment with a versioned query language, a query that looks right can be invalid against the current grammar. So the agent fetches get_query_dsl and get_graph_schema each session and composes every query from that, instead of hand-writing one that might silently fail.
• An empty result is ambiguous. Orbit's index lags real time, and an unindexed branch returns an empty graph rather than a real zero. We had the agent check the graph status first, so it reports an index gap instead of a misleading "nothing depends on this."
• Duo won't tell you your token usage. We wanted to show the token savings Orbit gives an agent, but Duo's dashboards only show adoption counts, and token data is admin-only billing aggregate. Rather than ship a number we could not honestly measure, we cut the claim and leaned on the real CI result instead.

Accomplishments that we're proud of

• Orbit does genuine work in three of the four skills. Blast radius, import frequency, and grounded subgraph extraction are real graph traversals, not prompt dressing. The agent asks the graph instead of crawling the repo.
• Every headline number is real. The scorecard is generated from CI job logs, not LLM estimates. We made safety a gate rather than a fudged weight, so the improvement number stays fully measured.
• Grounded generation that won't hallucinate dependencies. Because the fixes are built from the exact subgraph Orbit returns, the agent uses only symbols that actually exist. If something it needs is missing, it writes a // MISSING marker instead of inventing an import.
• One set of skills, two experiences. You can step through each skill in chat to understand the system, or launch the Slingshot flow to run the whole loop end-to-end.
• It's honest about its own limits. The agent refuses to open an MR if no variant survives the safety gate, and it flags an unindexed branch instead of reporting a misleading zero blast radius.

What we learned

• Grounding beats cleverness. An agent handed a precise subgraph makes far better decisions than one crawling raw files, and it stops inventing dependencies once you frame the dependency set as closed.
• A knowledge graph is only as good as your schema discipline. Most of our query bugs were not logic errors. They were us assuming a property or edge that Orbit does not have. Fetching the real schema before every query type fixed almost all of them.
• On Duo, the description is the routing. Writing a skill description as when to trigger rather than what it does is what makes the agent load the skill body instead of just following the one-liner.
• Measure-first changes the conversation. Once the numbers come from a real benchmark, "is this faster" stops being an argument and becomes a table.
• Honest scope makes a stronger project. Cutting the token claim we could not measure sharpened the story around the one thing we can prove, the real CI result.

What's next for Slingshot

• More languages. The CI runner is the only language-specific piece; the loop itself is language-agnostic. JavaScript/TypeScript and Go runners are next.
• Whole hot paths, not single functions. Use Orbit's impact scores to rank a service's functions, then run the loop across the top offenders automatically.
• An actual optimization search. Feed bottleneck_type and the real before/after numbers back in to generate a second round of variants — not just three shots, but a guided search.
• Catch it before it merges. Run Slingshot on merge requests so AI-generated code is profiled and blast-radius-checked before it ships, not after it's slow in production.
• Track debt over time. Persist scorecards so a team can watch measured performance debt get paid down release over release.

Built With

• git
• gitlab
• gitlab-ci-cd
• gitlab-duo
• gitlab-knowledge-graph
• gitlab-orbit
• markdown
• python
• yaml