Director's Memory

An agentic AI filmmaking assistant that detects continuity conflicts, asks whether changes are intentional, and remembers the director’s intent to guide future decisions.

Comment 1

Inspiration

AI filmmaking is becoming remarkably good at generating individual shots, but long-form productions still face a deeper problem: AI forgets what the director already decided.

A character's hair, wardrobe, props, lighting, or other visual details may drift from one scene to another. But not every change is an error — some changes are intentional creative decisions.

Director's Memory was inspired by one question:

How can an AI filmmaking system remember the director's approved creative intent across scenes without replacing the director's authority?

Our answer is to give filmmaking agents a persistent, structured memory of approved production decisions, including when each decision becomes active and when it expires.

What it does

Director's Memory is a persistent creative-decision memory and continuity system for agentic filmmaking.

It:

Uses Gemini to analyze the current scene and extract structured visual attributes. Retrieves the director's active creative decisions from ClickHouse Cloud. Uses the official ClickHouse MCP server and its run_query tool at runtime. Applies temporal scope so only decisions relevant to the current scene are retrieved. Compares the current scene against approved director memory. Detects continuity conflicts. Shows the underlying creative decision behind each conflict. Generates a suggested continuity fix while keeping the human director as the creative authority.

The core principle is:

Gemini observes. ClickHouse remembers. The Continuity Engine judges.

How we built it

Director's Memory is built as a lightweight agentic filmmaking prototype with a clear separation between observation, memory, and decision logic.

Gemini analyzes the current scene and returns structured attributes such as hair, wardrobe, props, lighting, location, and time of day.

ClickHouse Cloud stores approved directing decisions as persistent structured memory. Each decision contains fields such as character, attribute, approved value, effective scene range, status, source, and reasoning.

For runtime memory retrieval, the application uses the official ClickHouse MCP server.

The production memory path is:

Director's Memory ↓ Vercel API ↓ MCP Client ↓ Official ClickHouse MCP ↓ Runtime Tool: run_query ↓ ClickHouse Cloud

The /api/get-decisions endpoint queries ClickHouse through MCP as its primary path and exposes:

"memory_transport": "mcp"

so the runtime integration can be verified directly.

We also retain a direct ClickHouse backend fallback for resilience if the MCP service becomes temporarily unavailable.

The frontend and serverless API are deployed on Vercel, while the official ClickHouse MCP service runs on Render.

Challenges we ran into

One of the biggest challenges was distinguishing between simple memory and temporally valid creative memory.

A decision should not affect every future scene forever. For example, Maya's red coat may be approved only through Scene 9. Scene 8 should retrieve it, while Scene 10 should not.

We therefore had to design memory around:

Effective scene ranges Active versus expired decisions Attribute-specific retrieval Character-specific retrieval Supersession Safe fallback behavior

Another challenge was integrating the official ClickHouse MCP server as a true runtime path rather than using it only for a health check.

We also had to solve deployment and connection issues across Vercel, Render, and ClickHouse Cloud while keeping secrets server-side.

Accomplishments that we're proud of

We are especially proud that Director's Memory now demonstrates a real end-to-end persistent memory workflow.

In production:

Scene 8 retrieves four active director decisions through the official ClickHouse MCP runtime path:

Short black hair Red coat Silver shoulder bag Cold blue nighttime lighting

The API confirms:

"memory_transport": "mcp"

Scene 10 also successfully queries ClickHouse through MCP, but returns:

"decisions": []

because all four decisions expired after Scene 9.

This demonstrates that the system does not merely remember creative decisions — it understands when they should and should not apply.

We are also proud that Gemini, ClickHouse memory, and the continuity engine remain clearly separated instead of allowing the AI model to become the creative source of truth.

What we learned

The biggest lesson was that continuity is not simply visual sameness.

Good filmmaking often contains intentional change.

A useful filmmaking agent therefore needs more than image comparison or chat history. It needs structured knowledge of:

What the director approved When the decision became active When it expires Which character or attribute it applies to Why the decision exists

We also learned that persistent memory becomes much more powerful when it is queryable and scoped rather than stored as unstructured conversation history.

Most importantly, AI should not become the authority over creative intent.

The director decides. The system remembers.

What's next for Director's Memory

Next steps include:

Multi-character continuity memory Director approval and override workflows Camera and lens decisions Costume and prop lifecycle tracking Location and set continuity Automatic screenplay-to-memory ingestion Cross-scene visual verification Production approval history Multi-agent filmmaking workflows Shared creative provenance across production agents

The long-term vision is for Director's Memory to become a shared persistent creative-memory layer that multiple filmmaking agents can consult before taking autonomous production actions.

Generative AI knows how to create the next scene. Director's Memory helps it remember the film it is already making.

Built With

Share this project:

Updates