AutoEdit

UI interface
Project management

Inspiration

The inspiration for AutoEdit came from my own struggles with video editing. I had ideas and meaningful footage that often went unnoticed because professional editing requires considerable time and technical skill. I wanted to build an editor that could understand the story and personality in footage, rather than simply remove silence and join random clips.

What it does

AutoEdit creates a complete, context-aware video edit from raw footage. It analyzes visuals, dialogue, audio, chronology, orientation, pacing, and narrative structure, then produces a professional first cut with dialogue continuity, music synchronization, captions, reframing, B-roll decisions, and an editable Studio timeline.

How we built it

AutoEdit uses a completely local media pipeline:

FFmpeg for decoding, rendering, transitions, audio mixing, and export
Apple Vision for visual analysis, faces, composition, and orientation review
Local Whisper/Faster-Whisper for transcription and word timing
Ollama for optional local story reasoning and editorial planning
NumPy and signal processing for motion, shot boundaries, audio energy, and beat detection
Remotion for validated local motion-graphics assets
Python for the editing engine and a browser-based Studio interface

Codex and GPT-5.6 were used during development to analyze the codebase, design editorial systems, debug rendering issues, and improve the local-first architecture.

Challenges we ran into

One of the biggest challenges was preserving context across multiple clips. We combined Apple Vision, speech analysis, audio analysis, and Ollama, but early versions evaluated clips too independently. The middle of a sequence could lose its relationship to the opening and ending.

We also encountered chronology errors, incorrect video orientation, preview glitches, audio discontinuities, over-aggressive silence removal, and pacing that felt mechanical. We addressed these with chronology-first planning, narrative time budgets, speech-aware trimming, orientation verification, visual continuity checks, and professional J/L audio editing.

Accomplishments that we're proud of

We are proud that AutoEdit is built around local technology. Video, audio, transcripts, project files, and AI analysis remain on the user's device, which is important for private, personal, and unreleased footage.

We are also proud of the professional editing foundation: chronology preservation, context-aware pacing, local AI reasoning, orientation correction, source-color preservation, smooth preview rendering, audio continuity, and an editable Studio workflow.

What we learned

We learned that professional editing is not simply about removing silence or choosing the highest-scoring clips. Strong editing depends on chronology, performance, anticipation, reactions, visual continuity, emotional pacing, and knowing when to preserve an imperfect moment.

We also learned that AI needs persistent editorial context. Each clip must be understood as part of a larger narrative rather than as an isolated file.