REDLINE

AI writes the code. REDLINE attacks it, repairs it, and verifies the evidence before you trust it.

Comment

REDLINE — AI Proposes. The Sandbox Executes. Evidence Decides.

Inspiration

AI coding agents are becoming increasingly capable of writing code, fixing bugs, and implementing features. But an important question remains:

How do we know that the code generated by an AI is actually correct?

A patch can pass existing tests and still contain hidden edge cases, unexpected behavior, or security vulnerabilities. An AI agent can confidently declare that a problem is fixed without having sufficient evidence to support that conclusion.

I wanted to explore a different approach to AI-assisted software engineering.

Instead of building another coding assistant that simply generates code, I started building REDLINE — an autonomous software remediation and verification platform designed to challenge AI-generated solutions before accepting them.

The central idea is simple: an AI system should not blindly trust its own output. It should attempt to break its solution, learn from observed failures, repair the implementation, and evaluate the available evidence.

What it does

REDLINE is a multi-agent software engineering system that follows a controlled workflow:

BUILD → ATTACK → REPAIR → VERIFY

It consists of five specialized logical agents:

  • Architect: Analyzes the engineering task and prepares an implementation plan.
  • Builder: Generates the initial code implementation.
  • Red Team: Attempts to discover edge cases, counterexamples, and potential weaknesses.
  • Repairer: Uses observed failures to generate corrective patches.
  • Judge: Explains the outcome based on available execution and verification evidence.

A central orchestrator manages the workflow, validates agent outputs, and controls state transitions.

The system separates AI reasoning from actual execution:

  1. AI agents propose code and testing strategies.
  2. The sandbox executes the generated code.
  3. Machine-observed results are collected.
  4. Verification logic evaluates the available evidence.
  5. The Judge explains the outcome.

REDLINE also generates structured evidence receipts containing execution information, patch lineage, and integrity hashes.

Its SHA-256 hashing mechanism helps detect evidence modifications. It is not a digital signature or proof of software correctness.

The objective is to make software remediation more observable, auditable, and evidence-driven.

How we built it

REDLINE was developed as a full-stack application combining AI agents, backend orchestration, sandboxed execution, and a web-based interface.

Technology stack:

  • Backend: Python, FastAPI
  • Frontend: Next.js, React
  • AI infrastructure: Nebius Token Factory
  • AI model: NVIDIA Nemotron
  • Execution environment: Docker
  • Testing: pytest
  • Evidence integrity: SHA-256

The central orchestrator coordinates the five logical agents through an explicit state machine. Agents return structured outputs, while the orchestrator validates those outputs and decides which operation happens next.

For code execution, REDLINE uses Docker isolation with security restrictions such as:

  • Disabled network access.
  • Dropped Linux capabilities.
  • Non-root container execution.
  • no-new-privileges.
  • Restricted temporary storage and execution resources.

The verification architecture follows a fundamental separation:

[ \text{AI Proposal} \rightarrow \text{Sandbox Execution} \rightarrow \text{Observed Evidence} \rightarrow \text{Deterministic Evaluation} ]

This prevents an AI-generated explanation from being treated as independent proof of correctness.

Challenges we ran into

Building REDLINE involved several challenges beyond connecting language models.

1. Multi-agent orchestration

Coordinating multiple agents without allowing unpredictable agent decisions to control the entire workflow was challenging. We addressed this through centralized orchestration and explicit state transitions.

2. Trustworthy verification

One of the biggest challenges was distinguishing between an AI-generated claim and an actual machine-observed result. A model saying that code is correct does not make it correct.

We designed the workflow to rely on executable test results and structured evidence rather than model confidence alone.

3. Secure code execution

Executing AI-generated code introduces security risks. We implemented Docker-based isolation and restrictive container settings to reduce the potential impact of untrusted code.

4. Evidence integrity

Preserving the relationship between repository snapshots, patches, and execution results required a structured evidence system. We implemented SHA-256 hashing to make unauthorized changes detectable.

5. Integration and reliability

Integrating the frontend, backend, AI providers, Docker execution, and automated testing required repeated debugging, integration testing, and system-level validation.

Accomplishments that we're proud of

We are proud to have developed a working prototype that brings together autonomous AI agents, adversarial testing, isolated execution, and evidence-driven verification in one system.

Some of our key accomplishments include:

  • Implemented a five-agent architecture with centralized orchestration.
  • Integrated NVIDIA Nemotron through Nebius Token Factory.
  • Built a Docker-based sandbox execution workflow.
  • Implemented the BUILD → ATTACK → REPAIR → VERIFY cycle.
  • Developed structured evidence receipts with SHA-256 integrity checks.
  • Established patch lineage and execution tracking.
  • Created a full-stack dashboard for interacting with REDLINE and inspecting execution results.
  • Developed automated backend and frontend test suites.
  • Preserved a versioned benchmark with 84 integrity-checked artifacts.

The reported test results include 619 passing backend tests, 129 passing frontend tests, and successful Docker-based rehearsal runs. These are software and integration test results, not measurements of universal vulnerability-detection accuracy.

We are particularly proud of the architectural separation between AI-generated proposals and machine-observed execution evidence.

What we learned

REDLINE taught us that building an AI agent is not simply about selecting a powerful model or writing a sophisticated prompt.

The difficult part is designing a reliable system around the model.

Throughout this project, we learned about:

  • Multi-agent architecture and orchestration.
  • State-machine design and structured agent outputs.
  • LLM integration and prompt engineering.
  • Docker sandboxing and execution security.
  • Automated and adversarial testing.
  • Evidence integrity and reproducible workflows.
  • Full-stack application development.
  • The limitations of AI-generated code and automated verification.

Perhaps the most important lesson was this:

A trustworthy engineering system should be able to say "I don't have enough evidence" instead of confidently claiming success.

A successful test execution is evidence of a particular behavior under particular conditions. It is not a mathematical guarantee that software is free from defects.

What's next for REDLINE

REDLINE is currently a working prototype, and there is substantial room for improvement.

Our next development priorities are:

1. Stronger adversarial testing

Expand the Red Team's ability to discover meaningful edge cases, security weaknesses, and unexpected interactions.

2. Deeper verification

Introduce more comprehensive security checks, invariant evaluation, and independent verification mechanisms.

3. Reproducible benchmarking

Evaluate REDLINE against a collection of intentionally vulnerable repositories and compare two configurations:

[ \text{Baseline} = \text{Builder} ]

[ \text{REDLINE} = \text{Builder} + \text{Red Team} + \text{Repair} + \text{Verification} ]

Measure repair success, remaining vulnerabilities, false acceptance rate, execution time, and token consumption.

4. Improved evidence and transparency

Make verification scope, failed checks, executed tests, and unresolved findings visible to developers.

5. Developer workflow integration

Explore GitHub pull-request integration and continuous integration pipelines so that REDLINE can evaluate proposed code changes before they are merged.

Our long-term vision is to make adversarial verification a natural part of AI-assisted software engineering.

We believe the future of coding agents should not be measured only by how quickly they generate code, but also by how effectively their work can be challenged, tested, and verified.

REDLINE — Don't just generate code. Challenge it before you trust it.

Built With

  • agentic-ai
  • ai-agents
  • automated-testing
  • code-generation
  • code-repair
  • css
  • cybersecurity
  • docker
  • fastapi
  • git
  • html
  • llm
  • multi-agent-systems
  • nebius-token-factory
  • next.js
  • node.js
  • nvidia-nemotron
  • pytest
  • python
  • react
  • rest-api
  • sandboxing
  • sha-256
  • software-verification
  • typescript
Share this project:

Updates

posted an update —

REDLINE: From an Idea to an Agentic Software Remediation Platform

I'm excited to share an important milestone in the development of REDLINE, our project for the Nebius x NVIDIA Global AI Hackathon.

The Problem

AI can generate code, identify potential vulnerabilities, and suggest fixes. But how do we know whether those fixes actually work?

An AI-generated solution can look convincing while introducing new bugs, breaking existing functionality, or failing to address the original problem.

I wanted to explore a different approach: What if AI-generated code had to prove itself through execution and verification?

Introducing REDLINE

REDLINE is an autonomous software remediation and verification platform built around four stages:

BUILD → ATTACK → REPAIR → VERIFY

The platform coordinates five specialized logical agent roles:

  • Architect — Plans the solution.
  • Builder — Generates implementation proposals.
  • Red Team — Examines potential weaknesses.
  • Repairer — Proposes corrective changes.
  • Judge — Provides an advisory assessment.

What We Built

Some of the key engineering components include:

  • Python and FastAPI backend.
  • Next.js and React frontend.
  • Nebius Token Factory integration for NVIDIA Nemotron.
  • Docker-based isolated execution environment.
  • Automated testing using pytest.
  • SHA-256-based evidence integrity receipts.
  • Model-provider fallback and offline deterministic modes.

One of the central architectural decisions was to separate AI-generated recommendations from actual code execution and verification.

AI Proposes. The Sandbox Executes. Evidence Decides.

What I Learned

Building REDLINE taught me that developing agentic AI systems is much more than connecting an LLM to an application.

It involves system architecture, orchestration, model integration, security, sandboxing, testing, debugging, and understanding the limitations of AI-generated outputs.

Most importantly, I learned that an AI model saying something is correct is not the same as proving that it works.

What's Next?

REDLINE represents an important step in my journey into AI engineering and autonomous software systems.

I'm continuing to explore how agentic systems can become more reliable, secure, and useful for real-world software engineering.

This is still a learning journey, and there is much more to build, test, and improve.

Building, breaking, repairing, and learning — one iteration at a time.

REDLINE #Nebius #NVIDIA #Nemotron #AgenticAI #SoftwareEngineering #AIEngineering #Hackathon

Log in or sign up for Devpost to join the conversation.

Submission history