Website Front Page
How it Works (Part #1)
How it Works (Part #2)
How it Works (Part #3)
How it Works (Part #4)
Our Product
Results Page

Inspiration

When a program is compiled, much of what makes its source code understandable disappears. Variable names, abstractions, and high-level algorithmic structure are reduced to machine instructions. But the underlying computation is still there.

We wanted to explore a question: Can a neural network recognize what compiled code is doing by comparing it to implementations of known algorithms, without ever seeing the original source?

This idea has applications in reverse engineering, digital forensics, and binary analysis, where understanding unfamiliar software often means working without access to its source code.

That became CodeTrace, our attempt to build an actual learned binary-similarity system from the ground up, rather than simply handing assembly code to an LLM.

What it does

CodeTrace analyzes x86-64 assembly and identifies potentially related LeetCode problem implementations.

Users upload assembly code, and our pipeline extracts its functions, converts their instructions into structured representations, and passes them through a trained neural encoder. Each function becomes a 128-dimensional embedding, which we compare against a database of reference implementations using cosine similarity.

The system returns ranked candidate problems with similarity scores. A Gemini-powered explanation layer then interprets those results for the user.

Importantly, Gemini never sees the assembly itself. It receives only the candidate names and similarity scores produced by our neural model. The actual retrieval happens independently, without relying on an LLM to recognize the code.

Our backend also supports assembly files containing multiple functions, analyzing their representations separately rather than assuming an entire file contains one algorithm.

One of our favorite demonstrations involved Single Number, a LeetCode problem whose classic solution relies on XOR and bit manipulation. The model retrieved related implementations, and Gemini was able to construct an explanation that reflected meaningful algorithmic connections using only the retrieval results. We found this especially interesting because interpreting the raw assembly directly had been considerably less successful in our own testing.

How we built it

Building the dataset

One of our biggest tasks was creating the training corpus.

We started with a collection of C++ LeetCode solutions and developed a compilation and extraction pipeline to produce GCC -O0 x86-64 assembly. The resulting dataset contained:

4,906 assembly samples spanning 3,659 LeetCode problems
Extracted solution variants for problems containing multiple implementations
Anonymized function and symbol names
Validation of assembly structure, relocations, and disassembly

We then developed a frontend that converts assembly instructions and their operands into structured basic-block representations suitable for neural networks.

Our initial experimental protocol qualified 2,573 samples (52.45%). We deliberately excluded ambiguous or unsupported representations rather than treating every assembly file as a valid training example.

Training the model

We trained our models using supervised contrastive learning.

Each training batch contained eight LeetCode problems with two implementation variants each. The model was encouraged to produce similar embeddings for different implementations associated with the same problem while distinguishing implementations belonging to other problems.

We experimented with several neural architectures, including GINE, GATv2, an instruction-Transformer/GINE hybrid, and GraphGPS.

Surprisingly, a simpler encoder without explicit graph message passing proved competitive. Our final selected model uses structured instruction features, basic-block pooling, residual MLP layers, and a normalized 128-dimensional output.

It contains just 281,728 trainable parameters.

We also explored richer supervision using official LeetCode topic tags, source-derived algorithm annotations, and natural-language semantic distillation with MiniLM, BGE, and Qwen embedding models. Although several approaches produced promising individual results, their improvements weren't consistent across random seeds, so we retained our strongest validated baseline.

Building the application

We connected the inference system to a full-stack application using a FastAPI backend, database persistence, and a frontend for submitting code and exploring results.

The backend handles assembly processing, neural inference, top-k retrieval, score storage, and Gemini-generated explanations. We designed the system so that the neural retrieval component operates independently of the explanation layer.

Challenges we ran into

Dataset construction was far more complicated than we expected. Many C++ files contained multiple solution classes, helper functions, complex dependencies, and compiler-generated code. Separating implementations while preserving valid assembly required substantial work on symbol handling, relocation, and extraction. We also encountered concurrency bugs when parallel workers attempted to publish identical cached assembly objects.

Turning assembly into consistent neural inputs was another major challenge. We needed to preserve useful instruction and control-flow information while avoiding meaningless identifiers, addresses, and source-level metadata that could leak problem identity.

Model selection challenged our original assumptions. We initially expected graph neural networks to be the strongest architecture. Instead, our experiments showed that additional architectural complexity did not consistently improve retrieval. Semantic supervision produced similarly mixed outcomes, reinforcing the importance of reproducible comparisons rather than selecting whichever experiment happened to score highest.

Finally, integrating everything into a working application introduced its own problems: Python dependencies, model serialization, matching inference-time preprocessing to training, handling assembly files with multiple functions, backend communication, and connecting neural results to the frontend and Gemini.

Accomplishments that we're proud of

Built a 4,906-sample compiled-code dataset and an end-to-end neural retrieval pipeline.
Trained and evaluated five neural encoder architectures alongside deterministic baselines and multiple auxiliary-supervision approaches.
Achieved 55.9% Recall@1, 66.1% Recall@5, and 0.607 MRR on our selected model's validation benchmark, retrieving from 512 candidate problem IDs.
Achieved 35.7% Recall@1 and 57.1% Recall@5 on a separate held-out benchmark using an earlier checkpoint, with 31.2% higher MRR than our structural graph baseline.
Measured 2.8 ms median neural encoding and ranking latency, excluding assembly preprocessing and application overhead.
Built a functioning full-stack workflow that connects assembly analysis, neural retrieval, stored results, and AI-generated explanations.

We're particularly proud that the final system performs its similarity search using a compact neural model rather than outsourcing the core task to a general-purpose LLM.

What we learned

Our biggest lesson was that better model architecture doesn't automatically mean better performance.

We spent considerable time studying graph representations and experimenting with increasingly sophisticated architectures. But under our training conditions, a relatively simple instruction-based encoder performed competitively.

We also learned how important the training objective is. Two solutions to the same LeetCode problem can use entirely different algorithms, while two different problems can share nearly identical computational patterns. Recognizing algorithmic similarity and identifying an exact problem ID are related, but not identical, tasks.

Our experiments with topic tags and language-model-derived supervision reinforced this distinction. More semantic information is not automatically better unless the learning objective uses it effectively.

Finally, we learned the importance of honest evaluation. Our benchmark uses a fixed compiler configuration and weakly supervised solution variants, and not every corpus sample qualified for evaluation. Our system produces similarity rankings, not formal proofs of equivalence or calibrated probabilities.

Those limitations also give us a clearer roadmap for future improvements.

What's next for CodeTrace

We want to extend CodeTrace beyond its initial LeetCode-based retrieval setting toward more general binary analysis.

Our next directions include broader compiler and optimization-level support, more independently verified implementations, improved multi-function and whole-program reasoning, and direct ingestion of unfamiliar executable binaries.

We're also interested in continuing our semantic-distillation experiments, particularly using stronger reasoning models to generate implementation-specific supervision and learning which relationships between programs are most useful to preserve.

Longer term, we'd like CodeTrace to support both binary-to-binary similarity and natural-language-to-binary search, making it possible to locate implementations of particular algorithms or computational behaviors inside unfamiliar software.

Ultimately, CodeTrace is a step toward making compiled programs searchable by what they compute, rather than merely by the bytes or symbols they contain.