SegSense – Agentic Debugger for C Memory Bugs

8
1
7
5
3
6
4
2

Inspiration

We're second-year students, and C is the language we live in. If you write C, you know the feeling: the program runs fine on your machine, you submit it, and then it segfaults somewhere you've never seen. Or worse, it doesn't crash, and quietly prints garbage.

AddressSanitizer helps a lot. It tells you exactly where memory went wrong. But the report is a wall of hex addresses and stack frames, and it never tells you why, or what the fix should be. Meanwhile, AI models are great at explaining code, but you can't really trust a patch until you've seen it run.

So we thought: what if we put the two together? Let the sanitizer be the judge, let NVIDIA Nemotron do the thinking, back every fix with real secure-coding references, and keep going until the sanitizer is actually happy. That's SegSense.

What we learned

The biggest lesson was that an AI fix is only as trustworthy as whatever checks it. SegSense only accepts a patched program $p'$ when

$$ \text{accept}(p') \iff \text{it builds} \;\land\; \text{ASan finds nothing} \;\land\; \text{the output is unchanged} $$

Once we had that rule, we stopped worrying about the model "sounding confident". Either the sanitizer agrees, or the patch doesn't count.

We also enjoyed watching the model reason about bugs we'd made ourselves. One of our test programs transposes a $3 \times 5$ matrix with the wrong stride. ASan only says "heap-buffer-overflow at line 9". The real reason is that the code writes as far as index

$$ (\text{cols}-1)\cdot\text{cols} + (\text{rows}-1) = 4 \cdot 5 + 2 = 22, $$

into a buffer that only holds $15$ ints. Nemotron worked that out and changed a single cols to rows. In another example it noticed that $13!$ is bigger than $2^{31}-1$ and switched int to long long. Those are exactly the fixes we'd want a good teaching assistant to suggest.

A few smaller lessons:

Using two model sizes (a quick one to look first, a strong one to write the fix) was faster and cheaper than sending everything to the big model.
Every memory bug already has a name. Use-after-free is CWE-416, and the SEI CERT C rule that prevents it is MEM30-C. Pointing both the model and the user to those official pages made the fixes easier to trust and to learn from.
Sanitizers behave differently on clang vs gcc, and on Linux vs macOS. We learned that by getting burned, which you'll see below.

How we built it

The heart of SegSense is a simple loop:

Compile and run the program with AddressSanitizer and UndefinedBehaviorSanitizer turned on.
If it crashes, Nemotron 3 Nano takes a quick first look and writes a short note on what's going on.
At the same time, the Tavily search API looks up the official CWE entry and the SEI CERT C rule for that kind of bug, searching only trusted sites (MITRE CWE and SEI CERT C).
Nemotron 3 Ultra reads the code, the crash report, Nano's note and those references, explains the root cause, and writes a fixed version of the file.
We compile and run the fix under the same sanitizers. If it still fails, the new error goes back to Ultra and it tries again.

Both models run on Nebius Token Factory. Because its API is OpenAI-compatible, our whole client is about 100 lines of plain Python, with nothing to install. The Tavily search runs in parallel with Nano's triage, so it adds no waiting time, and if Tavily is ever unavailable SegSense simply fixes the bug without references. You can use SegSense from the terminal (segsense fix prog.c), or open the web demo and watch each step stream in live, with the reference links shown next to the patch. We added a test suite that runs on every push, and deployed the demo with Docker.

The part we're proudest of: on all 8 bug types we tested, from overflows and use-after-free to double free, leaks and integer overflow, Nemotron got the fix right on the first try, and every bug was matched to its correct CWE entry and CERT C rule. Each fix took about 3–10 seconds.

Challenges we faced

Honestly, most of our time went into things that looked like they worked but didn't:

The model IDs we found online were wrong. The Nemotron names in third-party lists weren't the ones Token Factory actually serves. We added a segsense models command that asks Token Factory directly, and switched to the Nemotron 3 family.
The model sometimes thought so hard it never answered. Nemotron 3 reasons before replying, and with a small token limit it could use the whole budget thinking. We gave it more room, and made SegSense say clearly when that happens.
Ultra copied our line numbers into its code. We number the lines so the model can match them with the sanitizer's report. On our first real run, it pasted those numbers into the fix, and 4 of 6 patches didn't even compile. A clearer instruction, plus a small cleanup step, brought that to zero.
Search results were messy at first. Our first Tavily queries returned huge PDFs, dead pages and snippets full of website menus. Splitting the lookup into two targeted searches, one for the CWE entry and one for the CERT C rule, and keeping only the official page from each, gave the right two links for all 8 bugs.
A fake "clean run" on a teammate's Mac. Apple's version of the sanitizer refuses to start when leak detection is on. SegSense saw no crash report and happily said "clean run". Now leak checks only run where they're supported, and any sanitizer error counts as a failure.
Leaks that clang just couldn't see. Our GitHub tests kept failing on the memory-leak example, but only with clang. It turned out a leftover copy of the pointer on the stack made the leak look "still in use". Telling LeakSanitizer to ignore stale stack memory took detection from $0/50$ runs to $50/50$.
Hosting. Many cloud containers block what LeakSanitizer needs to run, which would have broken every demo request. SegSense now notices that at startup and quietly turns leak checks off, so everything else keeps working.

Each of these bugs taught us something about the very tools SegSense is built on, which felt fitting for a project about debugging.