TextSequence

Inspiration

I started TextSequence from a simple question: what would a video editor look like if humans and AI agents could safely work on the exact same timeline?

Most AI video tools either generate media or sit outside the editor and suggest what a human should do. I wanted to explore something different: an open-source NLE where an external AI agent could inspect a real project, understand its tracks and clips, and perform actual editorial operations through MCP.

I only found out about Build Week from an email on July 17th, so that is when I started building.

What it does

TextSequence is a lightweight, local-first, MCP-native non-linear video editor.

Humans can edit through a visual React interface, while external AI agents can access the same canonical timeline through MCP.

TextSequence currently supports:

Multi-track video timelines Local media import Clip splitting, trimming, moving and deletion Cross-track editing and compositing Timeline markers Deterministic FFmpeg rendering and MP4 export Immutable revision history Revision diffs Forward-only restore Atomic multi-operation transactions EditGuards for protecting clips, ranges and projects from conflicting edits 27 MCP tools and 8 MCP resources

The core idea is that there is no separate "AI timeline." The human interface, REST API and MCP clients all operate on the same project state.

How we built it

TextSequence uses a Python/FastAPI backend, a React/TypeScript frontend, FFmpeg and ffprobe for local media processing, and MCP over Streamable HTTP for agent access.

I also developed a three-role AI-assisted workflow during the hackathon:

Sol acted as the architect, reasoning through major system changes before implementation. Luna handled implementation, testing and human acceptance verification. Terra acted as an adversarial release reviewer whose job was to find problems before anything was merged.

Each major feature went through architecture, implementation, real application testing and adversarial review.

I pushed that process hard enough to use 100% of my available OpenAI usage, right up to the final Terra evaluation of v0.5.0. Because of that, I used Qwen for the final MCP demonstration, which unexpectedly became a nice demonstration of the architecture itself: TextSequence is designed to be agent-agnostic.

Challenges we ran into

One of the biggest challenges was evolving TextSequence from a single-track prototype into a real multi-track editing model without breaking existing projects or immutable revision history.

Introducing schema v3 meant older schema-v2 revisions had to remain cryptographically valid in their original stored form. I had to separate raw revision authentication from in-memory schema migration so historical revisions could be safely loaded without rewriting them.

Concurrency was another challenge. Once humans and agents can both modify the same timeline, stale revisions and conflicting edits become real problems. TextSequence uses revision checks, atomic transactions and EditGuards to make those interactions safer.

Multi-track rendering also required replacing a simple sequential FFmpeg pipeline with deterministic layered video composition and embedded-audio mixing.

The adversarial review process caught several subtle problems. In the final v0.5.0 review, Terra discovered that moving a clip vertically between tracks worked through the normal editor but was incorrectly rejected as NO_CHANGES when performed inside a transaction. The issue was fixed, regression-tested and reviewed again before the release was approved.

I also found bugs simply by using the editor myself, including clips displaying the first imported video's filename and timeline movement being too dependent on drag interactions. Both were fixed before release.

Accomplishments that we're proud of

I am most proud that TextSequence moved beyond being a proof-of-concept UI and became a working foundation for human-agent collaborative editing.

An external agent can inspect the actual timeline, discover stable clip and track IDs, and perform real editorial actions such as splitting clips. Those changes become part of the same revisioned project that the human is editing visually.

By the end of Build Week, TextSequence had reached v0.5.0 with:

A real multi-track video timeline 27 MCP tools 8 MCP resources Schema migration with preserved revision integrity Atomic transaction support Forward-only revision restore EditGuards Deterministic multi-layer rendering Embedded audio mixing 148 passing backend tests

The project is also fully open-source and does not require an API key for core manual editing, rendering or external MCP access.

What we learned

The biggest thing I learned is that making creative software "AI-native" requires much more than exposing UI buttons as tools.

Agents need stable identifiers, strict schemas, deterministic operations, reliable revision history, conflict handling and safe transaction boundaries.

I also learned the importance of making the human interface and agent interface share the same domain logic. Otherwise, you effectively build two editors that eventually disagree with each other.

Finally, the Sol → Luna → Terra workflow taught me how useful it can be to give AI systems distinct responsibilities instead of asking one agent to design, implement and approve its own work. The adversarial review stage repeatedly found issues that happy-path implementation testing missed.

What's next for TextSequence

Development will pause until the Build Week results on August 12th, but there is plenty I want to explore next.

Near-term improvements include polishing timeline interaction and fixing smaller preview UX issues.

Longer-term, I want to explore:

Editable audio tracks Better agent-friendly MCP schemas and tool discovery More advanced timeline editing operations Transitions and compositing controls Transcript-aware editing Semantic media understanding AI-assisted shot selection Proxy workflows for larger projects

The broader goal remains the same:

build an open editing layer where humans and AI agents can safely collaborate on the same creative project.

Built With

• codex
• fastapi
• ffmpeg
• mcp
• nle
• openai
• python
• react
• typescript