MODEL ON LIMITED DATA
MODEL ON LIMITED DATA
FINAL MODEL
FINAL MODEL

Inspiration

From Dean and Isayah's experience in customizing AI model for hardware design (caid-technologies/parti-base), we’ve trained models for several agentic pipelines. We wanted to explore whether agents could continuously improve by learning from traces generated during previous jobs—turning successful agent behavior into training data for smaller, specialized models.

What it does

EmergeGPT creates a MCP server to connect to Craft, a data quality pipeline product, access the Data Catalog, uses the data from the tables to curate reasoning traces from the teacher model, then captures the traces, transforms them into a structured training dataset, and uses LoRA fine-tuning to adapt a smaller model to similar tasks.

This creates the foundation for a continuous learning loop: agents perform tasks, their traces become training data, and smaller models become increasingly capable within a focused domain.

How we built it

We integrated the Emergence MCP server into an agentic workflow and created sample jobs involving the GitHub repositories dataset. We then:

Collected traces from the agents’ tool use and responses.
Cleaned and converted those traces into fine-tuning examples.
Prepared the examples as a structured dataset.
Applied LoRA fine-tuning to a smaller language model.
Evaluated whether the adapted model could reproduce the patterns learned from the original agent runs.

Challenges we ran into

The largest challenge was converting raw agent traces into useful training examples. Traces contain tool calls, intermediate reasoning, errors, and inconsistent output formats, so deciding what to retain required careful preprocessing.

We also had to balance dataset quality against quantity. A smaller set of clean, representative traces was often more valuable than a larger collection containing noisy or redundant interactions.

Accomplishments That We’re Proud Of

We built an end-to-end prototype that connects agent execution with model improvement. Rather than treating traces as disposable logs, EmergeGPT turns them into reusable training assets.

We’re especially proud that the approach can transfer behavior from a more capable agentic system into a smaller, more efficient model through LoRA fine-tuning.

What We Learned

Agent traces can contain valuable supervision, but they are not automatically training-ready. Trace selection, normalization, and formatting are essential parts of the learning pipeline.

We also learned that MCP provides a useful interface for generating grounded, tool-assisted examples. Combining those examples with parameter-efficient fine-tuning opens the door to specialized models that are faster and less expensive to operate.

What’s Next for EmergeGPT

Next, we want to complete the continuous-learning loop by automatically evaluating completed jobs, selecting high-quality traces, updating the training dataset, and periodically fine-tuning new model versions.

We also plan to:

Compare fine-tuned models against their base models using repeatable benchmarks.
Expand beyond GitHub repository data to additional tools and data sources.
Add safeguards for filtering sensitive or low-quality trace data.
Explore automated trace scoring and failure-driven training.
Deploy specialized models back into agentic pipelines and measure improvements in cost, latency, and task completion.