TakeOne

Inspiration

AI can describe a cinematic shot in seconds. Turning that description into real footage still takes cameras, lighting, carefully timed movement, and people to coordinate it all.

We felt that gap while filming with friends. We could imagine a moving reveal or a dramatic close-up, but executing it meant coordinating the camera, lighting, and performance ourselves. Our ideas were bigger than the crew we had.

Meanwhile, we had two SO-101 robotic arms confined to a desk and an RC truck that needed better controls. The arms could articulate but could not travel. The truck could travel but needed intelligence.

That sparked a question: what if describing the shot was the first step toward physically filming it?

We combined a mobile base with independent camera and lighting arms, then connected them to AI direction, simulation, and editing. TakeOne grew from the ambition to give small teams the tools to bring their cinematic ideas into the real world.

What it does

TakeOne is an AI-powered robotic film studio that helps solo creators and small teams go from idea to finished video.

Describe your vision, and the AI Director translates it into a script, shot list, framing, camera movement, lighting, and performance cues. Rehearse the shots in a 3D simulator, refine their paths and timing, then approve them for filming.

A mobile cart carries independent camera and lighting arms, coordinating movement through the scene with camera angle, light positioning, and lens cues. A separate tracking mode helps follow and frame performers.

After filming, recorded takes enter the built-in editor for arranging clips, trimming shots, adjusting pacing, adding transitions and effects, and exporting. AI editing proposals help refine the sequence.

TakeOne turns creative direction into a rehearsed shot, a recorded take, and an edited final video.

How we built it

Hardware: A reinforced RC truck carries two SO-101 arms with camera and light holders. An ESP32 receives USB serial commands, while the drivetrain motor controllers communicate over CAN bus.
Power: A DC-DC converter steps the 42 V battery down to a 12 V bus for the arms, control electronics, and laptop power adapter. Isolated USB I/O helps reduce ground-loop interference.
AI direction: The OpenAI Responses API generates structured scripts, shot plans, and editing proposals. GPT-Live powers conversational direction through constrained function calls.
Simulation: Python validates and compiles movement plans. MuJoCo supplies robot geometry and kinematics, while Three.js displays interactive rehearsals.
Tracking: OpenCV processes camera frames, MediaPipe detects body or face landmarks, and PID control converts framing errors into movement corrections.
Recording and editing: Blackmagic Camera integration controls iPhone recording and lens cues. React provides the editor interface, and FFmpeg renders previews and exports.

Control, simulation, and editing run locally, while generative AI features use cloud APIs. Physical execution requires operator approval.

Challenges we ran into

Electrical interference: High-current motors and shared grounds disrupted communication, requiring careful power distribution and isolation.
Limited connections: The cart, arms, camera, and controllers had to work together within USB-port, grounding, and latency constraints.
Control-loop latency: Camera processing, MediaPipe inference, and motor commands had to remain responsive despite noisy detections and delayed frames.
Creative intent to physical motion: Natural-language requests had to become precise paths, arm poses, and timing within the rig’s limits.
Simulation versus reality: Physical execution introduced factors beyond the preview, including traction, cables, and hardware response.

Accomplishments that we're proud of

Built a mobile rig with independent robotic camera and lighting control.
Connected AI direction, 3D rehearsal, supervised filming, and editing in one workflow.
Created 39 configurable movement presets and integrated live performer tracking.
Achieved zero observed brownouts and minimal UART packet drops during testing.
Integrated vision processing and PID control with active timeout watchdogs.

What we learned

AI helped interpret creative intent, while deterministic software handled geometry, timing, and movement limits. Separating those responsibilities made the system easier to test and control.

Standard hardware and interfaces helped us validate the complete workflow before investing in custom electronics. Digital commands made communication easier to inspect, log, and validate, while supporting explicit timeout behavior.

Most importantly, filmmaking demands more than reaching a position. Movement must serve the framing, performance, and timing of the scene.

What's next for TakeOne

Improve motion smoothness, performer tracking, and repeatability through measured physical testing.
Integrate power and control electronics into a custom board with a fitted touchscreen.
Move critical motor control and watchdog supervision to an RTOS-based embedded controller to reduce timing jitter.
Add environmental sensing and coordinated multi-camera filming, including behind-the-scenes capture, with drone coordination as a longer-term goal.