Inspiration
Long stories drift. A character's eyes are green in chapter 1 and blue in chapter 9. Someone buried in chapter 2 carries a crate in chapter 11. A broken wrist heals in a day. Writers and editors catch these slips only by rereading the whole manuscript, which takes weeks and costs money. We wanted a tool that does that first pass, and that a writer can actually trust.
What it does
StoryForge AI reads a manuscript chapter by chapter and flags contradictions. Every warning shows both quotes side by side, so the writer can check it in seconds and decide what to fix. The tool never rewrites the text. Beyond the Issues list it builds a character map, a story bible, a timeline, and an Ask box that searches the book by meaning.
How we built it
The AI only reads. Our own code does the checking.
- Read: an AI model reads each chapter in passages of about 700 characters and writes down facts: who, which attribute, what value, and the exact quote.
- Link names: rules decide that "Nell" and "Nell Harker" are the same person.
- Check in code: the story clock (ages and dates), traits (eye colour, scars, left or right) and lasting states (dead, broken, destroyed, lost).
- Confirm: a contradiction becomes a warning only when the code confirms it.
The backend is Python with FastAPI, the database is PostgreSQL with pgvector, the frontend is plain JavaScript, and the AI is free NVIDIA-hosted models. It has email and password sign-in with optional Google sign-in, runs in Docker, and is deployed on Render and Neon.
Challenges we ran into
- Free AI only. A local model took 5 to 12 minutes per chapter. One provider retired its model, and another's free plan allowed only 8,000 tokens a minute. We settled on a single NVIDIA model.
- Consistency. Mixing models changed the readings between runs, so one model reads every chapter.
- False alarms. An AI checker and AI name linking added false alarms and long timeouts, so we moved the judgement into code.
- A hidden bug. A stale API key in the operating system silently overrode our
.envfile and caused 401 errors.
Accomplishments that we're proud of
In every measured test run the tool raised no false alarms. We measured by writing stories with mistakes planted on purpose and an answer key, then scoring the warnings against the key.
| Test set | Planted mistakes found | False alarms |
|---|---|---|
| Stories we tuned against | 21 of 36 (58%) | 0 |
| Night Shift (not tuned) | about 4 of 10 | 0 |
| Kestrel (not tuned) | 2 to 4 of 9 | 0 |
What we learned
A writer's trust depends on precision, so we chose to miss some mistakes rather than invent any. Letting the AI read while code decides made every warning explainable. Recall on a story we had never seen is roughly 25 to 40%, and we say so plainly.
What's next for StoryForge AI
- A faster, stronger model to raise recall, with a separate "Worth a look" tier kept apart from confirmed warnings
- Checks for contradictions inside one chapter, in dialogue, and for characters known only by a role
- A larger test set with stories written by people outside the team
- Export of the Issues list, and a way to mark a warning as intended
- A paid hosting plan so the site never sleeps
Built With
- css
- docker
- fastapi
- html
- javascript
- neon
- nvidia
- pgvector
- postgresql
- python
- render