Codex Submission Doctor

01_codex_submission_doctor_cover
06_automated_test_verification
05_self_scan_results
03_corrected_project_results
02_incomplete_project_scan
04_self_scan_start

Codex Submission Doctor Inspiration

Hackathon submissions often fail for surprisingly small reasons: a missing README.md, an incomplete license, a broken test command, unclear setup instructions, or files that were accidentally left out of the repository.

These problems are easy to overlook when a developer is focused on finishing the actual product. They are usually discovered only during the final submission review, when there is very little time left to fix them.

That inspired Codex Submission Doctor: a local pre-submission diagnostic tool that examines a project repository, identifies potential submission problems, and explains what should be fixed before the project is submitted.

The idea was to create something that feels less like a rigid linter and more like a doctor performing a structured health check on a project.

Privacy was also an important design principle. A repository may contain unfinished work, configuration details, internal documentation, or other sensitive information. For that reason, Codex Submission Doctor analyzes the selected project locally and does not upload the repository contents to an external service.

What It Does

Codex Submission Doctor runs a structured inspection of a project and classifies its findings into clear categories:

Blockers — issues that may prevent a valid or complete submission Warnings — problems that should be reviewed but may not stop submission Passed checks — requirements that were successfully verified Unavailable checks — checks that could not be performed in the current environment

The tool can inspect areas such as:

repository structure required documentation license and metadata files test availability and execution configuration quality dependency information project readiness

Instead of presenting developers with an unstructured wall of terminal output, it produces a readable project health summary and an overall score.

It can also generate machine-readable and human-readable reports, including:

terminal reports JSON reports HTML reports

A self-scan mode allows Codex Submission Doctor to inspect its own repository using the same rules it applies to other projects.

How We Built It

Codex Submission Doctor was built as a command-line application using Python.

The application is organized around a diagnostic pipeline. Each check evaluates one specific part of the repository and returns a structured result containing information such as:

check name status severity explanation recommended action

These individual results are then collected and transformed into a complete project health report.

The terminal interface uses Rich to provide readable formatting, status indicators, structured panels, and visually distinct categories. This makes it possible to understand the most important problems quickly without manually searching through long logs.

The reporting layer separates the analysis data from the presentation format. The same diagnostic result can therefore be rendered as terminal output, JSON, or HTML without rerunning the complete inspection.

We also created automated tests for the diagnostic logic and report-generation components. These tests helped verify that blockers, warnings, successful checks, and unavailable checks are classified consistently.

The development workflow included using Codex CLI to support implementation, code review, debugging, test creation, and repository analysis.

Challenges We Faced Designing useful severity levels

One of the biggest challenges was deciding which findings should be treated as blockers and which should only produce warnings.

For example, a missing required project file may justify a blocker, while a short description or missing optional configuration may be better represented as a warning.

If every issue is treated as critical, the report becomes noisy and stressful. If too many issues are treated as minor, the tool may fail to prevent an incomplete submission.

We addressed this by defining clear result categories and keeping the diagnostic logic separate from the user interface.

Handling different project structures

Repositories can be organized in many different ways. A Python application, a JavaScript project, a documentation repository, and a mixed-language project may all require different checks.

The tool therefore needed to avoid assuming that every repository follows one exact template. Checks are designed to report when something is unavailable rather than incorrectly marking every unsupported situation as a failure.

Running tests safely

Automatically executing project commands can introduce risks. A diagnostic tool should not silently run arbitrary operations without the user understanding what is happening.

Codex Submission Doctor follows an explicit, user-initiated workflow. The inspection starts only after the user requests it, and checks that cannot be performed safely or reliably can be marked as unavailable.

Producing reports for both humans and tools

A terminal report should be quick to read, while a JSON report should be structured for automation. An HTML report should be understandable to reviewers who may not use the command line.

Supporting all three formats required us to create a shared internal result model rather than embedding diagnostic logic directly into one output format.

Testing the doctor with the doctor

The self-scan feature was both useful and challenging. It required the tool to evaluate its own repository objectively and exposed inconsistencies that might not have appeared when testing only artificial sample projects.

This became an effective form of dogfooding and helped improve the reliability of the checks.

What We Learned

Building Codex Submission Doctor reinforced several important lessons.

First, developer tools are most useful when their output is actionable. Simply stating that something failed is not enough; the developer should also understand why it matters and what can be done next.

Second, severity classification is part of the product experience. Clear distinctions between blockers, warnings, successful checks, and unavailable checks make the report easier to trust.

Third, structured internal data makes a tool much easier to extend. Because diagnostic results are separated from presentation, new output formats and checks can be added without redesigning the entire application.

We also learned that self-analysis is extremely valuable. Running the tool against its own repository helped uncover edge cases and forced us to experience the workflow exactly as another developer would.

Finally, privacy and user control should be designed into developer tools from the beginning rather than added later. Local inspection and explicit scan initiation became core parts of the project rather than optional features.

Accomplishments

We are especially proud that Codex Submission Doctor can:

perform a structured project health inspection distinguish blockers from non-critical warnings provide clear and actionable diagnostic messages generate terminal, JSON, and HTML reports run automated tests for its core behavior inspect its own repository through self-scan mode keep repository analysis local provide developers with a final readiness score

The result is a practical tool that can be used as the final checkpoint between finishing a project and submitting it.

What’s Next

Future versions could include:

configurable submission profiles for different hackathons support for additional programming languages and frameworks custom repository rules CI/CD integration GitHub Actions support automatic fix suggestions comparison reports between multiple scans submission-readiness history plugin-based diagnostic checks

The long-term goal is to make Codex Submission Doctor a reusable project-readiness assistant that helps developers catch preventable submission problems before reviewers ever see them.

Final Result

Codex Submission Doctor turns the stressful final review of a hackathon repository into a clear and repeatable process:

Scan the project, understand the problems, fix the blockers, and submit with confidence.