SpecGuard — Make Code Keep Its Word

💡 Inspiration

Every developer has experienced the frustration of writing code that runs successfully but doesn't behave as expected. Traditional testing requires manually creating test cases, identifying edge cases, and debugging failures, which can be time-consuming and difficult.

We asked ourselves a simple question: What if developers could describe what their code should do, and AI could help verify whether it actually does it?

That question inspired SpecGuard, an AI-powered Python verification platform designed to bridge the gap between software requirements and actual code behavior.

Our goal was to make software testing more accessible, efficient, and transparent, helping developers spend less time finding bugs and more time building reliable applications.

🚀 What It Does

SpecGuard transforms natural-language specifications into executable test cases and verifies whether Python functions behave as intended.

Users simply provide their code and describe the expected behavior. SpecGuard then:

Understands Requirements: Uses AI to extract testable conditions from natural-language specifications.
Generates Test Cases: Creates deterministic tests covering expected behavior and relevant edge cases.
Executes Code: Runs supported Python functions and compares actual outputs against expected results.
Detects Issues: Identifies failing requirements, unexpected behavior, and potential code problems.
Suggests Fixes: Provides AI-assisted recommendations for correcting identified issues.
Retests Improvements: Allows users to verify proposed fixes against the original test cases.
Generates Reports: Presents clear pass/fail results, requirement coverage, and supporting execution evidence.

Unlike tools that simply generate code or suggest improvements, SpecGuard focuses on verifying whether the implementation actually fulfills its intended requirements.

🛠️ How We Built It

We built SpecGuard as a full-stack web application that combines modern frontend development, backend orchestration, AI-assisted analysis, and automated code execution.

Tech Stack

Frontend: React and TypeScript
Backend: Python and FastAPI
Database: SQLite
AI Models: NVIDIA and Nebius model providers
Code Execution: Docker-based isolated execution
Production Security: gVisor-based sandboxing architecture
Deployment Infrastructure: Caddy and HTTPS

Development Process

We started by designing a workflow that connects written specifications directly to executable tests.

The frontend provides an interactive workspace where users can create projects, submit Python code, describe requirements, and inspect verification results.

Our FastAPI backend manages authentication, project storage, AI provider communication, and test execution.

The AI component interprets specifications and generates structured test cases. These cases are executed against the submitted Python functions, and the resulting observations are compared with expected behavior.

We also developed a reporting system that links test outcomes to individual requirements, making it easier to understand failures and review suggested code changes.

To improve reliability, we implemented execution limits, error handling, project ownership checks, and a retesting mechanism.

⚡ Challenges We Ran Into

1. Interpreting Natural-Language Requirements

Specifications aren't always precise. Developers may describe expected behavior without mentioning important edge cases or constraints.

Designing a system that converts these descriptions into meaningful, testable requirements was one of our biggest challenges.

2. Making AI-Generated Tests Reliable

AI can generate useful test cases, but its predictions aren't automatically correct.

We had to distinguish AI-generated expectations from actual execution evidence and ensure that incomplete or malformed results weren't incorrectly marked as successful.

3. Executing User Code Safely

Running submitted Python code introduces security risks, including infinite loops and excessive resource consumption.

We addressed this through bounded execution and container-based isolation, while preparing a stronger production architecture using gVisor.

4. Integrating Multiple Components

Connecting the frontend, backend, AI providers, execution engine, and reporting system required consistent data structures and careful error handling.

Ensuring that each component communicated reliably was essential to making the complete verification workflow function.

🏆 Accomplishments That We're Proud Of

Our biggest achievement is developing a functional end-to-end AI-assisted verification platform, rather than stopping at a prototype that only generates test cases.

We successfully implemented:

Automated extraction of testable requirements.
AI-assisted test generation and Python verification.
Detailed reports showing passed, failed, and incomplete evaluations.
Suggested fixes with the ability to retest changes.
Persistent project storage and user authentication.
A responsive interface with light and dark themes.
Local Docker execution tests and a live NVIDIA-powered demonstration.

We're particularly proud of building SpecGuard around a principle of evidence-based verification.

Instead of assuming that AI-generated answers are correct, our platform checks the actual behavior of the submitted code and presents the results transparently.

📚 What We Learned

Building SpecGuard gave us valuable experience in combining artificial intelligence with traditional software engineering practices.

We learned how to integrate language models into structured workflows, transform natural-language requirements into executable tests, and manage code execution through a backend service.

We also strengthened our understanding of frontend development, API integration, database management, containerization, authentication, and application security.

One of our most important lessons was that AI-generated results are only as useful as our ability to validate them.

We also learned that passing a set of generated tests doesn't guarantee perfect software. Meaningful verification requires clear specifications, reliable execution, careful interpretation, and transparent evidence.

🔮 What's Next for SpecGuard

Our immediate priority is completing production deployment, including secure hosting, gVisor runtime validation, monitoring, and operational testing.

Beyond that, we envision expanding SpecGuard with:

Multi-Language Support: Verification for Java, C++, JavaScript, and other languages.
GitHub Integration: Import code directly from repositories.
CI/CD Automation: Run specification-based checks automatically when code changes.
Improved AI Testing: Generate stronger edge cases and identify ambiguous requirements more effectively.
Team Collaboration: Let developers review specifications, test results, and suggested fixes together.
Enhanced Reporting: Provide deeper insights into verification coverage and recurring failures.

Our long-term vision is to make specification-driven verification a natural part of everyday software development.

Because writing code is one thing. Making sure it keeps its word is another.