SFT Fine Tuning Data Studio for Highly Customizable LLMs

The Problem:

Fine tuning an LLM sounds great, and should be easy. There are several low cost managed training pipelines available, and deployment options abound on user friendly sites such as featherless Ai, Nebius and hugging face. The problem model quality. Without quality training data, the model is going to be a flop. High quality training data is the key. But where do you find it? When most of the data available is stock or bulk, that doesn’t translate to a customized, personal or domain specific model. Decentralized, user-sovereign intelligence is becoming more and more sought after and necessary, and accessible fine tuning is more relevant than ever. SFT fills a gap in both the accessibility and quality of fine tuning in a never before click through UI that takes any user from idea to fully tuned and deployed custom LLM.

Inspiration :

I built this tool because I needed to use it. After filling up several fine tuning workshops with enthusiastic ready to learn students, I realized training data was no longer a personal bottleneck. Newer versions of LLM models have upscaled features but highly downgraded performance when it comes to generating the example turn datasets required for SFT fine tuning. In addition to my workshops, I am involved in important alignment research and badly needed an tool for quality fine tuning data. I couldn’t find one. So I built SFT studio pro and kept building until it became the best fine tuning app on the market.

What it does.

SFT Studio pro opens and asks you chose a mode or open a saved project. The modes are automated or manual.

Automated mode asks a few questions and customizes the project for the user, first by defining the project, then suggesting a plan for training topics, running the data generation engine, explaining the document converter and triad alignment options, and automating the quality assessment gate. Once the data set is ready, the user can choose to export the data set or initiate training. They can choose from 3 base retrained models (Qwen 4B, Qwen 9B, or LLama 70B). The system formats the final dataset and runs the training pipeline, updating the UI on job status as it runs. When training is complete, the user is given an option to download weights, a link to the model ready to go on Hugging face, links and instructions to various deployment options, and a free 15 minute first chat hosted right on the app interface.

Manual mode is a great choice for more experienced developers who know what a SFT training data set is made of and want full control over the data set. Each step of the process can initiate automation, and the training pipeline is available in this mode as well.

The first step is the project definition. A system prompt, a target goal, a golden example and a list of rules and constraints for your model, and then choose instruct or conversational training format. When you’re ready to commit, place a permanent lock on project definition to ensure data consistency. Next, the batch synthesizer. Here batches of training on a particular skill or knowledge topic are created under the project definition with their own batch specific temporary rules including a batch description (i.e. humor, tool calling, mathematics etc) and any special instructions and restrictions. Batches can be generated in quantities of 5, 10, 25, 50, 100, 200 or 250. Available formats are single turn, multi turn and reasoning. A progress bar shows job status. Once complete, the examples are displayed in readable format. They can be individually edited or deleted, and the whole batch can be rejected or approved to send to the QA Que There is also a document converter. User uploads documents and specifies knowledge, literature or transcript and fills in instructional input. The system analyzes the document and reports recommended batch synthesis. User may approve for generation or edit instructions and re-analyze. Generations are synthesized and displayed for edit or approval to QA Que. Next is an optional brand alignment system for ethical grounding based on compelling research and evidence. It is not mandatory. If chosen, pre-generates examples are formatted to project specifications and added to data registry.
The QA gate protects the data registry from error or drift analyzing each example, processing one batch at a time and assigning a score. Any examples flagged for drift, corrupted text, broken template or instruction compliance are presented with explanation for each flag along with high, medium or low severity warning. User can edit each manually or ignore warning. The most common choice is to deduplicate, a one click fix that deletes all flagged occurrences and passes the remaining quality examples, or “regenerate flagged” feature that repairs each error and adds flagged examples back to the batch. The data registry will not accept batches until they have been assessed by the QA gate. The last step is to choose final format (seven commonly used SFT templates across instruction and conversational modes available) and convert the data set into final form with a 10% validator split ready for download in JSON for human audit or JSONL ready for the training pipeline. Automated training is optional.

How we built it;

I built the app initially on Vertex but migrated to Replit for variety in generation models used via API to generate diverse data sets. The data synthesizer uses a separate api call for every example, orchestrated across ten models on digital ocean inference. three formats (single turn, multi turn and reasoning) and 7 templates. For every batch, a topic planner runs in the backend and assigns a unique subtopic to each example to avoid redundancy and provider cache repetition. The automated pipeline uses a Together AI skill and CLI to train the model and export to bugging fave, and the 15 minute first chat feature creates a one time dedicated endpoint on together AI’s infrastructure.

Challenges we ran into:

Oh my God! People think vibe coding is easy. It is not easy on a project like this. 😭😭😭 Coding agents in replit have a dedicated passion for fixing things that aren’t broken 16 miles from assigned scope. I had to watch the agents build every single second with my finger on the halt button. Also, instruction an agent to instruct another agent to make examples that instruct an agent became like a nightmare at times. Me: Make the reasoning template produce legitimate, high functioning traceable train of thought reasoning outputs. Gemini (after a very long work session) “I have written a hard rule for my future instructions to use a long, traceable and thorough train of thought reasoning process and respond to you in a json array at all times (response is written in json, nothing has changed in the api instructions) 🤪😖😫😩

Accomplishments that we're proud of:

The app is remarkable. It works fantastic and the data sets produce very well trained models that match and often exceed user intention. It takes something daunting like fine tuning and makes it accessible to anyone. Instead of using low quality LLM outputs, downloading bulk generic data, or trying to hand type training data yourself, fine tuners can now easily create exactly what they need and envision for shaping model behavior. It’s also web and mobile based, not terminal, so you can work on it while riding the bus to work, lying in bed or even sitting on the toilet. It’s easy and accessible- a first for fine tuning data generation.

What we learned:

I learned how to use very precise, clear and direct prompts for a vibe coding agent. My vibe coding before this app was often vague, poorly worded or left room for too much agent drift. Now I say exactly what the agent needs to hear and there’s no room for misinterpretation or nuance in order execution. I also learned how amazing it feels to build something that is a first of its kind and a highly needed tool for independant developers and enterprise deployers of LLMs when the commercial models don’t meet the need. It’s a legitimately useful tool. It’s a great accomplishment.

What's next for SFT Dataset Studio Pro I will used it extensively for my alignment research projects and I’m confident in the level of quality it offers machine learning developers. Next is marketing to the broader industry, opening doors to Soverign LLM production for business and individuals at the consumer level.

Built With

• digitalocean
• firebase
• fireworks
• gemini
• google-cloud
• nebius
• postgresql
• replit
• togetherai
• vertex