Mimic

Inspiration

Animating a 3D character by hand is slow. Every movement has to be posed and keyframed, and a few seconds of natural motion can take hours. Professional motion capture solves this, but it needs marker suits, multi-camera studios or sensor suits that cost thousands of dollars, which puts it out of reach for students, indie developers and small creators.

We asked a simple question: what if your laptop webcam was enough? Stand in front of the camera, pick a character, and watch it move exactly like you. That idea became Mimic.

What it does

Mimic is a Blender add-on that turns an ordinary webcam into a motion capture studio:

Bring any character. Paste a link to a rigged model (.fbx, .glb, .gltf, .vrm, or a .zip). Mimic downloads it, imports it and saves it into the Blender project. Google Drive, Dropbox and GitHub share links are converted automatically.
Automatic bone mapping. It recognises Mixamo, VRM, Rigify and Unreal-style skeletons, so most characters work with one click.
Character select screen. Choose who you want to be. The chosen character fills the screen on its own and copies your movements live. Press C to switch characters, Esc to finish.
Record and export. Capture your performance as keyframes and export it as FBX for Unity, Unreal or any other tool.
Private by design. All tracking runs locally. No video ever leaves your computer.

How we built it

We started with a browser prototype (three.js, three-vrm, Kalidokit and MediaPipe for the web) to prove the idea, then moved to Blender, where animators actually work.

Mimic is written in Python using Blender's bpy API. Each frame:

OpenCV captures a webcam frame.
MediaPipe Pose Landmarker detects 33 body landmarks in 3D, in metres.
The landmarks are converted from MediaPipe's camera coordinates (x right, y down, z away) into Blender's (character facing $-Y$, $Z$ up):

$$ (x, y, z){\text{MediaPipe}} \;\rightarrow\; (x,\; z,\; -y){\text{Blender}} $$

Retargeting. For each limb, we take the direction between two joints (shoulder → elbow, hip → knee) and find the smallest rotation that swings the bone from its current direction $\mathbf{d}{\text{cur}}$ to the tracked direction $\mathbf{d}{\text{target}}$:

$$ \mathbf{q} = \text{rot}(\mathbf{d}{\text{cur}} \rightarrow \mathbf{d}{\text{target}}), \qquad \theta = \arccos(\mathbf{d}{\text{cur}} \cdot \mathbf{d}{\text{target}}) $$

The hips and head get a full orientation built from two axes (left–right and up), so the character can turn and tilt. We walk the skeleton from parent to child so every bone inherits its parent's motion correctly.

Each new rotation is blended with the previous frame using quaternion slerp, and can be recorded as a keyframe on Blender's timeline.

Challenges we ran into

Jitter. Raw webcam tracking trembles constantly. A simple moving average removed the shake but made the character lag behind. We switched to the One Euro Filter, which adapts its smoothing to how fast you move:

$$ \tau = \frac{1}{2\pi f_c}, \qquad \alpha = \frac{1}{1 + \tau / T_e}, \qquad f_c = f_{c,\min} + \beta \,\lvert \dot{\hat{x}} \rvert $$

When you hold still, $f_c$ stays low and the character is steady. When you move fast, $f_c$ rises and the character keeps up. We also added a deadzone that ignores bone changes under about $2^\circ$, and extra filtering on depth, which is the noisiest axis from a single camera.

Flickering limbs. When an arm or leg sat at the edge of the frame, it switched rapidly between "tracked" and "not tracked". We added hysteresis: a limb turns on above 65% visibility and only turns off below 35%.
Every rig is different. Mixamo calls it mixamorig:LeftUpLeg, VRM uses J_Bip_L_UpperLeg, and Rigify uses DEF-thigh.L. We wrote a name normaliser that strips prefixes and matches dozens of naming patterns. A single wrong match (a thigh mapped to a hand!) taught us to keep every mapping visible and editable.
macOS camera permissions. Blender doesn't ask macOS for camera access, so the webcam failed silently. Launching Blender from Terminal lets it borrow Terminal's permission.
Dependencies inside Blender. Blender ships its own Python, so MediaPipe and OpenCV had to be installed into it from a button inside the add-on.

Accomplishments that we're proud of

Full-body motion capture with nothing but a webcam, no suits, markers or extra hardware, running in real time inside Blender.
Works with almost any character. Users paste a link and Mimic handles downloading, importing, bone mapping and saving.
Smooth, usable motion. Combining the One Euro Filter, deadzones and visibility hysteresis turned shaky raw tracking into animation steady enough to record.
A real "pick your character" experience. The character select screen and full-screen stage mode make it feel like a finished product, not a tech demo.
Privacy-first. Everything runs on the user's own machine.

What we learned

How 3D skeletons really work: rest poses, parent–child transforms, and why quaternions beat Euler angles for smooth rotation.
Signal processing matters as much as the AI model. Good filtering made a bigger visual difference than a better tracker would have.
Building tools for creators means designing for people who aren't programmers: clear error messages, one-click setup and sensible defaults.
How to package Python dependencies for a host application like Blender, and how operating-system permissions can quietly break things.

What's next for Mimic

Hands and face. Finger tracking and facial expressions such as blinking, mouth movement and eyebrows, which MediaPipe already supports.
Moving through space. Hip translation so characters can walk, jump and crouch.
Multiplayer mimicry. Tracking several people at once, each driving their own character.
Auto-rigging. Letting users bring models without a skeleton and rigging them automatically.
Live streaming mode. Sending Mimic's output to OBS for VTubers and live performers.