Codex Skill Miner

Discover reusable AI engineering workflows from successful Codex sessions using deterministic Git and test evidence. Offline-first, reproducible, and built for trustworthy workflow reuse.

Comment

Inspiration

AI-assisted coding sessions often disappear after a task is complete, taking successful engineering practices with them.

Codex Skill Miner explores a simple question: Can successful AI-assisted engineering workflows be discovered, validated, and eventually reused instead of being forgotten?

It is an offline-first developer tool for developers and teams that want to preserve successful AI-assisted engineering practices. In the bundled demonstration example, Codex Skill Miner reviews the git repository (Codex Skilll Miner) and identifies 6 canonical sessions, discovers 4 exact workflow fingerprints, and qualifies 1 reusable workflow candidate supported by 3 independent Git commits.

What it does

Codex Skill Miner validates canonical session records, resolves only explicit full Git SHAs, fingerprints ordered activity sequences with SHA-256, groups exact workflow matches, and applies fixed recurrence, success, and completeness gates.

It produces deterministic evaluation.json, workflow.json, workflow.md, and workflow.html artifacts that are fully reproducible from the same evidence.

The runtime does not use an LLM, infer Git relationships, rank workflows, or automatically generate reusable skills.

How we built it

The project was built as an offline-first Python application using:

  • Python 3.10
  • Pydantic v2
  • Typer
  • Git
  • SHA-256 workflow fingerprinting
  • Pytest
  • Ruff
  • uv

The runtime has no network or OpenAI API dependency.

Human–AI Collaboration

OpenAI Codex accelerated implementation, testing, debugging, and milestone validation.

GPT-5.6 supported architecture reviews, documentation refinement, README improvements, edge-case analysis, and engineering critique.

Human review defined the research direction, qualification policies, project scope, architectural decisions, and final approval of every implementation milestone.

Every milestone emphasized deterministic behavior, reproducibility, and explicit evidence before any future AI enrichment.

Challenges we ran into

The primary challenge was preserving evidence integrity while keeping the implementation deterministic.

Only explicit full Git commit SHAs are accepted. Timestamp, branch, path, commit message, and fuzzy attribution are intentionally ignored to prevent inferred relationships.

We also isolated every qualification rule with focused tests to ensure duplicate commits, incomplete evidence, or post-validation failures could never falsely qualify a workflow.

Another challenge was producing both machine-readable and human-readable artifacts while ensuring every output originated from the same deterministic workflow evidence.

Throughout development, we documented architectural decisions, milestone reviews, engineering trade-offs, and lessons learned in our project JOURNAL.md.

Accomplishments that we're proud of

  • Built a fully offline-first developer tool with no runtime LLM or network dependency.
  • Designed a deterministic workflow qualification pipeline backed by explicit Git and testing evidence.
  • Created reproducible JSON, Markdown, and HTML artifacts from the same authoritative workflow evidence.
  • Demonstrated an end-to-end workflow using a bundled dataset that judges can reproduce locally in minutes.
  • Established a clear separation between workflow evidence, qualification, presentation, and future AI enrichment.
  • Developed the project through milestone-based human–AI collaboration while maintaining human ownership of research direction and engineering decisions.

What we learned

The biggest lesson was that trust comes from explicit evidence rather than interpretation.

Separating workflow evidence from presentation made the architecture easier to validate, extend, and review without affecting qualification correctness.

Human–AI collaboration proved most effective when AI accelerated implementation while humans retained responsibility for research direction, engineering policy, and final decisions.

This MVP suggests that successful AI-assisted engineering sessions can be represented as reusable engineering knowledge when their evidence, validation, and qualification policy are explicit, inspectable, and reproducible.

What's next for Codex Skill Miner

This MVP demonstrates deterministic workflow discovery and qualification from bundled engineering evidence.

Our next goal is to transform qualified workflow candidates into reusable AI skills while preserving the same evidence-first architecture.

Future work includes:

  • promoting qualified workflow candidates into reusable SKILL.md packages
  • mining workflows across multiple repositories and development teams
  • discovering higher-level engineering patterns beyond exact workflow matches
  • benchmarking workflow quality across different AI models and coding agents
  • building an ecosystem for discovering, evaluating, and sharing reusable AI engineering skills

We believe future AI engineering systems should not only generate code, but also accumulate validated engineering knowledge that developers and AI agents can confidently reuse.

Built With

  • ai-engineering
  • deterministic
  • developer-tools
  • git
  • gpt-5.6
  • odex
  • offline-first
  • python
  • workflow-mining
Share this project:

Updates