Re-Write Assist

Inspiration

I’ve been inspired by LLM writing for a while. Yet, I did not intend to enter into this hackathon. It was only after reflection from my time as a writing tutor in college that I decided to re-tool an LLM to help with writing style. The good of ‘good writing’ does not happen during the generation. It happens during the editing. It happens during the revisions. We need people to reflect on their writing.

Codex

You will see that this project is a continuation of my Prime-RL environments, of my evolutionary strategy training edit to the Prime-RL library and an extension of what I have done in the past. With direction, Codex was the primary deployer and trainer of models. I directed the size and final form factors but I did not man the GPUs directly. Additionally, I allowed Codex to generate the video. Essentially all of the code generated was from delegation. This written piece is directly from me. All models used were trained during the hackathon. The most important learning below was determined during the hackathon.

What I learned

I learned that small models in the LLM family are primarily lexical. This is true for both encoders and decoders. Jasper, a 600m parameter encoder and second on the public MTEB english leaderboard, is nearly 85% lexical. Models less than 4B in the decoder lineage are also primarily lexical. We don’t get to strong semantic qualities until the models are over 8B parameters. We required a minimum of a 12B parameter judge model for semantic similarity and found that Jasper was no better than an n-gram model for style rewards. I had Codex generate and test this knowledge during the hackathon.

For our tests, we generated 2 different benchmarks to test lexical and semantic understanding. Even in the 8B regime, models still failed ~20% of the time in determining if a low lexical overlap rewrite was semantically similar to an original piece of writing.

How I developed Re-Write Assist

Our environment works by perturbing LoRA weights and grading the generation in standard ES style. The output is a rewrite of the input text. It is graded on semantic alignment with the input and stylistic alignment with the target. The goal was to create a model which could write like a well regarded essayist (Paul Graham) but would not be such a good re-writer that it would turn AI prose into something that could pass leading detection tools. The submission achieves both. Most AI writing which is long enough does not get re-written to a quality that would pass detection (using Pangram). However, human writing also does not get re-written to a point where it fails as AI writing. The word choice aligns well with Paul Graham (even the percentage of sentences which begin with And or But).

I chose evolutionary training strategies (ES) because it is robust against entropy (a weakness of RL) and because I am considered GPU poor. ES allowed me to iterate quicker on my environment and train large-ish (over a couple B parameters) on a single GPU. I was able to perform training runs of a 30B model on a single H100. I have shown in the past that ES is comparable to RL in terms of efficacy but can have biases in how the model learns.

The final 12B parameter model (Gemma 4) was initially trained with SFT on a dataset pair of PG essays and LLM rewrites. This pair was generated by rewriting PG essays by paragraph. The rewrites were initially summarized before passing only the summary to an LLM for rewriting. The summary pass was modified multiple times in order to create a valid SFT dataset. Initial tests were underwhelming because our SFT was not ideal. The dataset was also originally used to create a centroid for stylistic grading. However, it was found an n-gram model outperformed our centroid created from the Jasper model (even with the final improved dataset).

I then used LLM essays (Huggingface dataset: CK0607/qwen3.5-9b-blogprovider-traces) to generate 21,000 rewrites from our ES trained 12B Gemma 4 model. I trained a 1B parameter rewrite model with SFT. The rewrite model used my prior adaptive warmup to select the learning rate and batch size. I used WSD where the decay was the Paul Graham-LLM pair dataset. This model was finally quantized and verified to meet performance thresholds. This final quantized model is the one released with local rewrite harness for the hackathon.

The style was lexically driven. I used an n-gram model trained on Paul Graham essays. The semantic grader required a large model. I used the 12B base (Gemma 4) with no LoRAs from which we were training to rewrite essays. Codex and I then experimented with a few different variations of an HTML frontend and model harness backend. Originally, I wanted to host the page and allow users to download the model locally. However, ONNX quantized models were not high enough quality. We finally ended with a GGUF variation and backend which could be installed from Github.

The future

When I train a similar model in the future, I will use mid-layer embeddings from the semantic grading model for stylistic representation. I originally used an encoder model. However, most off the shelf encoder models are too lexically driven to be useful. There is an opening in the OS ecosystem for a strong, true semantic model. However, our final semantic grader was strong enough to determine even low lexical rewrites as semantically equivalent. This model may be strong enough to determine deeper author qualities than lexically driven ones (which an n-gram captures).

Built With

• codex
• llama-cpp
• prime-rl
• python