SkillStack

Inspiration

SkillStack was inspired by a simple research question: can a language model keep learning new skills without constantly retraining one large monolithic model?

In many ML systems, adding new tasks often means fine-tuning again, risking catastrophic forgetting, higher cost, and harder debugging. SkillStack explores a different direction: keeping small frozen LoRA “skill frames” and selecting or composing them through a router.

The original project was already a continual-learning research prototype before OpenAI Build Week. During the hackathon, I wanted to avoid adding a superficial feature just for the submission. Instead, I used GPT-5.6 to review the research design and identify the most meaningful weakness to improve.

The key weakness was routing transparency. The project had “chain” routing behavior, but the hierarchy was not fully validated as true parent_id traversal. That became the Build Week extension.

What I built

During OpenAI Build Week 2026, I added a reproducible Build Week Routing Audit for SkillStack.

The extension compares four routing modes:

1. Flat routing — direct task selection.
2. Current hybrid-chain routing — the earlier chain-style behavior.
3. True parent-chain routing — validated traversal using explicit parent_id relationships.
4. Seeded random-chain routing — a deterministic random baseline for comparison.

The new Build Week work includes:

• a CPU-only routing audit script;
• true parent_id traversal validation;
• checks for missing parents and cycles;
• deterministic comparison between routing modes;
• unit tests for the audit logic;
• JSON and Markdown audit outputs;
• a semantic-shift benchmark schema validator;
• documentation that clearly separates pre-existing SkillStack work from new Build Week work.

The audit can be run without GPU training:

python -m unittest tests.test_build_week_routing_audit -v
python scripts/build_week_routing_audit.py --demo
python scripts/build_week_semantic_shift_validator.py data/build_week_2026/semantic_shift_schema.jsonl

How I built it

I used GPT-5.6 first as a research reviewer. Instead of asking it to simply add a feature, I asked it to analyze the existing project and identify where the research story was weakest.

The conclusion was that the routing hierarchy needed a clearer and more testable audit. That helped turn a vague research concern into a concrete engineering task.

Then I used Codex as an implementation partner. Codex helped implement the routing audit script, the semantic-shift validator, unit tests, documentation, and reproducible output files. I kept the scope intentionally small: the goal was not to fake a large new ML breakthrough, but to create a clean, honest, and testable extension that improves the project’s research credibility.

Challenges

The biggest challenge was separating old work from new Build Week work.

SkillStack already had a paper draft, earlier experiments, benchmark results, and scripts. For the hackathon, I needed to make sure the submission clearly showed what was built during Build Week. That is why the new documentation explicitly explains which parts existed before the hackathon and which parts were added during it.

Another challenge was keeping the extension reproducible. Full ML training can require GPU access, long runtimes, and fragile dependencies. For the Build Week extension, I chose a CPU-only audit pipeline so judges can run the new work quickly and verify the logic.

A third challenge was avoiding overclaiming. The demo output is a synthetic audit fixture, not a claim that the full research system reaches a specific production-level benchmark. The value of the extension is that it makes the routing logic more inspectable, testable, and honest.

What I learned

This project taught me that AI coding tools are most powerful when they are used for research iteration, not only for generating code.

GPT-5.6 was useful for identifying the right problem. Codex was useful for turning that problem into runnable code, tests, and documentation.

I also learned that a small, well-scoped extension can be more valuable than a large but unclear feature. For a research prototype, reproducibility and honesty matter a lot. The Build Week extension improves SkillStack by making one of its most important components — routing — easier to test and explain.

What's next

The next step is to turn the semantic-shift schema into a real benchmark with approved examples across tasks. After that, SkillStack can evaluate whether true parent-chain routing is more robust than flat routing when inputs become ambiguous, adversarial, or semantically shifted.

Longer term, SkillStack could become a lightweight research framework for experimenting with continual learning, adapter composition, routing, compression, and AI-assisted ML research workflows.

Built With

• codex
• continual-learning
• github
• gpt-5.6
• json
• lora
• machine-learning
• natural-language-processing
• python
• pytorch
• unit