Trace

Trace: Your partner for solving tricky coding problems.

Comment

Inspiration

As I was debugging my code for problem-solving challenges, I realized that even when the solution passed all the test cases, it did not mean I had understood the algorithm well. Code tends to fail on some edge cases, and sometimes it becomes quite tricky to see what exactly is going wrong in your logic. I thought of building something that could do more than just confirming whether the solution was right or not. Trace AI was born from this concept.

What it does

Trace AI is an AI-powered code mentorship platform that analyses Python solutions, creates more edge cases and helps in diagnosing errors in algorithmic reasoning. It leverages several AI models to create test cases and diagnoses misconception by the user. It also runs the code in a sandboxed environment and gives insightful feedback rather than just a wrong output.

How we built it

The system is split into an independent frontend interface and a high-performance evaluation backend:

  • Frontend: Developed with React & Vite to provide a fast, responsive user interface, hosted on Vercel.
  • Backend: Developed using FastAPI to manage the execution microservice pipeline, hosted on Render.

AI Orchestration Stack

We orchestrate three distinct models via the Nebius Token Factory to distribute specialized tasks efficiently:

  • Nemotron-3_5-Lightning: Used to generate adversarial edge cases based on the user's problem.
  • Nemotron-3.5-Super-VL: Used to analyse the specific algorithmic misconceptions within the code logic.
  • Nemotron Ultra 550B: Used to provide step-by-step Socratic hints in the interactive Trace tab.

3-Layer Execution Sandbox

To safely evaluate user code on our microservice, we engineered a custom security mitigation pipeline:

  1. AST Level Checks: Static analysis intercepts and blocks potentially dangerous constructions before they ever execute.
  2. Sanitized Subprocesses: Spawns code execution inside isolated child processes stripped of critical system environment variables.
  3. OS-Level Constraints: Enforces strict CPU time limits and memory caps to completely block infinite loops and resource exhaustion and also protects from Denial of service.

Optimization Note: I have implemented intelligent code execution short-circuits. The backend completely skips unnecessary calls to the AI models if the code successfully passes all tests natively or if the security layer flags dangerous code constructions early.

Challenges we ran into

One of the main problems that I faced while developing this system was the necessity to make the code execution safe without breaking correct execution of the valid Python solutions. As well as the problem of coordination of multiple AI models without making it too costly or time-consuming. I needed to take into account multiple kinds of test failures and make sure that the feedback is based on real execution results, not AI answers.

Accomplishments that we're proud of

I'm really pleased with having an opportunity to create a full-fledged product with frontend, backend evaluation pipeline, and its deployment. What I'm most proud of, is the reduction of LLM token usage by 82% on average, with 91.7% peak reduction, that I managed to achieve via early checks and short-circuiting by test results.

What we learned

I realized from this assignment that developing AI applications is not just about integrating an LLM into the interface. The aspects such as model orchestration, security, performance execution, latency, and tokens consumed have been identified. The significance of using test results in providing evidence before asking the AI to solve an issue has also been highlighted.

What's next for Trace

The next step is to expand the platform's multi-lingual capabilities to support languages like Java, C++, and Go, alongside deeper integration with external competitive coding platforms. I also plan to refine the generation engine to provide longitudinal analytics, tracking a developer's common logical pitfalls over time to offer personalized algorithm roadmaps.

Built With

Share this project:

Updates

Submission history