Continueml

image

continueml was inspired by a major limitation in today’s AI creative tools:

AI can generate anything, but it doesn’t remember anything.

Creators building characters, storylines, or visual worlds constantly struggle with inconsistencies faces change, styles drift, details break across generations. I wanted to solve this missing layer by giving AI a form of persistent memory that feels natural, automatic, and reliable.

What Inspired Me

While experimenting with episodic AI content and multi-character storytelling, I noticed that creators were spending more time rewriting context than actually creating. Every new generation required repeating long descriptions.

That sparked the core idea:

What if AI could store entity memories, recall them across generations, and enforce consistency automatically?

This became the guiding vision behind continueml.

What I Learned

Building continueml pushed me deep into the architecture of AI memory and semantic understanding. I learned how to combine:

Visual embeddings (CLIP) for appearance tracking
Semantic embeddings for personality, traits, and narrative context
Vector search for retrieving relevant memories
Prompt enhancement pipelines for context injection
World + branch versioning for alternate timelines
Consistency analysis for validating generated images against memory

It also taught me how creators think about continuity and how much value a memory layer adds.

How I Built It

I built continueml as a full-stack SaaS platform using:

Frontend: Next.js 16
Backend API: Node.js + tRPC
Entity Metadata: PostgreSQL
Vector Memory Storage: Pinecone
Generation Queues: Redis + BullMQ
Embeddings & Generation: OpenAI, CLIP, and Gen-3
Deployment & CDN: Cloudflare/Vercel

The architecture is modular, with dedicated systems for the Memory Engine, Prompt Enhancer, Consistency Analyzer, and Versioned Worlds.

Challenges I Faced

Some key challenges included:

Designing a reliable weighting system between visual and semantic embeddings.
Ensuring consistent prompt enhancement without making prompts overly long.
Handling branching worlds without duplicating too much vector data.
Building a conflict detection system for merging branches.
Ensuring the memory engine remained fast even with thousands of stored entities.

These challenges helped shape the final architecture and taught me a lot about building scalable AI infrastructure.