Living Image - limg

Turn one anime portrait into a portable, code-controlled character—no layers or manual rigging.

Comment

Inspiration

Live 2D Character animation usually starts with a layered PSD, manually placed landmarks, or a hand-built rig. That workflow works well, but it requires specialized skills and significant preparation before a character can move.

I wanted to explore a different question:

What if a favorite illustration could be compiled once into a portable character file, then controlled from any compatible app through the same API?

Living Image turns a supported near-frontal anime portrait into a reusable .limg character asset. Instead of producing a single video, it creates a file that can be saved, moved, reopened in a separate Viewer, and controlled programmatically.

What it does

The Compiler analyzes a PNG or JPEG and automatically detects the face, eyes, pupils, and mouth. It derives local deformation data, evaluates confidence, and produces one of three explicit outcomes:

  • Full: the character supports all validated controls.
  • Limited: unsafe controls are disabled, but the remaining character is still usable.
  • Reject: no character file is produced, and the reason is shown clearly.

A successful result is exported as a self-contained .limg file containing the source texture, geometry, capabilities, and behavior parameters.

The separate Viewer loads that file and controls the character through a common normalized API:

player.setState({
  blinkLeft: 0,
  blinkRight: 0,
  gazeX: 0.35,
  gazeY: -0.1,
  mouthOpen: 0.25,
  breath: 0,
});

The Viewer performs no per-frame machine-learning inference. Playback is local, deterministic, and model-free after compilation.

How I built it

Living Image is divided into three main parts:

  1. Compiler A Python pipeline performs anime-specific face detection and 28-point landmark analysis. It derives eye, iris, mouth, deformation, protection-mask, confidence, and capability metadata.

  2. Portable .limg format The generated character file stores image data, per-character geometry, behavior parameters, supported capabilities, and provenance information.

  3. Browser Runtime and Viewer A TypeScript Canvas runtime loads any compatible .limg and applies the same controls across different characters.

The hosted prototype uses a Cloudflare Worker and an on-demand private Python container. The Compiler and Viewer remain separate: after compilation, the generated .limg can be opened in the Viewer without uploading it again.

Codex was used throughout the project for technical research, implementation, test generation, failure reproduction, experiment documentation, and deployment packaging. Examples are preserved directly in the repository: the compiler and Runtime implementation, the browser-based motion and pixel-preservation tests, the documented research experiments and failure analysis.

I made the core product and engineering decisions, including treating .limg as a portable character asset instead of generating a one-off video, keeping manual landmark placement out of the primary workflow, separating model-backed compilation from deterministic playback, and rejecting unsupported images instead of forcing visibly broken results. These decisions are reflected in the .limg schema, the explicit full, limited, and reject outcomes, and the separate Compiler and Viewer architecture documented in the project README.

GPT-5.6 helped evaluate architecture choices, identify claims that were not yet supported by evidence, analyze visual failure modes, and refine the project into a reproducible submission rather than a one-image demonstration. The repository preserves this process through its validation reports, holdout testing, and submission documentation.

Challenges I faced

Preserving the original illustration

Naive image warping can make a face look like jelly, crush highlights, or distort nearby line art. The Compiler therefore creates bounded local deformation regions and protected eye masks.

The project also includes real Chromium Canvas tests that verify:

  • motion remains inside Compiler-authored feature regions;
  • protected pixels preserve their RGB values;
  • unprotected pixels still move;
  • blink transitions return to the original open state.

This was important because “something moved” is not enough. The movement must preserve the identity and visual style of the source image.

Knowing when not to animate

Automatic detection is imperfect. A one-eye-closed portrait, a very small face, heavy occlusion, or an unsupported art style can produce misleading geometry.

Instead of silently generating a broken asset, Living Image records capabilities and can disable gaze, mouth, or blink independently. Completely unsupported inputs produce no .limg.

One external holdout set also exposed a missing style-domain gate. I preserved that failure in the research results rather than tuning it away after seeing the test data.

Separating compilation from playback

The Compiler may use detection models, but the final character should not require those models every frame. This led to the Compiler–file–Runtime architecture:

Portrait
→ automatic analysis
→ portable .limg
→ separate Viewer
→ common Runtime API

This separation makes the generated character reusable in other web applications, native apps, games, or AI companion interfaces.

What I learned

The most important lesson was that automatic character animation is not only a rendering problem. It is also a file-format, capability, validation, and failure-handling problem.

I learned that:

  • a portable asset is more reusable than a generated video;
  • safe refusal is often better than visually broken output;
  • per-character geometry belongs in the compiled file, not in Runtime branches;
  • visual quality needs measurable browser-level tests, not only screenshots;
  • separating model-backed compilation from deterministic playback produces a cleaner and more extensible architecture;
  • small reactions such as blinking, looking, breathing, and responding can make a static image feel significantly more present.

What’s next for Living Image

The next step is to move Living Image from a validated prototype into a practical character format that other applications can adopt.

First, I want to improve the visual model. This includes more reliable iris and highlight separation, better eyelid occlusion, subtle head movement, hair motion, and additional expressions without sacrificing the original illustration’s identity. These improvements will continue to follow the same principle used in the current prototype: unsupported motion should be reduced or disabled rather than forced into a visibly broken result.

I also want to expand the interaction layer. A character should be able to follow the pointer, react to touch, respond to speech, and expose higher-level actions such as:

character.lookAt(x, y);
character.react("happy");
character.speak(audio);
character.touch({ x, y });

These interactions would make the character feel aware of the user rather than simply playing a predefined animation.

Another major goal is to formalize .limg as an open and documented portable format. The format should support versioning, capability discovery, provenance metadata, extensions, and consistent playback across web, desktop, mobile, games, and AI companion applications.

The Compiler also needs a broader and more clearly measured support domain. Future work includes testing substantially more rights-cleared images, adding stronger style and occlusion detection, comparing different deformation techniques, and introducing human evaluation for perceived naturalness and character identity preservation.

Finally, I want to integrate Living Image into StayWithYou, an AI companion project where a character can quietly watch, listen, speak, and react through small motions. The longer-term vision is not one animation tool or one application. It is a system where a user can bring a favorite image, compile it once, and continue using the same living character across different experiences.

The core direction remains:

Do not generate one animation. Compile a reusable capacity to animate more freely.

Built With

Share this project:

Updates