Inspiration

AI video tools can create beautiful results, but describing exactly how objects should move is still difficult. We wanted a more direct way to communicate motion: move a block, set a keyframe, and show the AI what you mean. Moblocks brings the hands-on control of an animation editor into an image-to-video workflow.

What it does

Moblocks turns an image or a frame from a video into a scene of editable 3D blocks. Creators can move, rotate, scale, and animate those objects on a timeline, then use the resulting motion clip and original image to guide AI video generation.

The editor also supports layered image composition, animated image cutouts, local project saving, and frame and video exports. Finished renders have shareable pages with synchronized before-and-after comparisons. We also implemented PayPal credit purchases so creators can fund paid AI operations through a familiar checkout flow.

How we built it

We built the browser editor with React, TypeScript, Three.js, and Vinext. Muse Spark identifies objects, SAM 3.1 segments them, and MoGe3 estimates camera and depth information to help place editable blocks in 3D. AI3D's H3 Max uses the authored motion clip and image reference to generate the final video.

Our animation engine handles transforms, keyframes, interpolation, and undo/redo. PayPal's Orders API powers credit purchases, with server-side payment verification, verified webhooks, and a credit ledger connecting purchases to AI usage.

Challenges we ran into

Turning a flat image into a useful 3D starting point required careful camera fitting and depth estimation. We also had to preserve the same framing and timing across the viewport, exported motion, and AI render input.

Paid operations introduced another challenge: interrupted requests and repeated callbacks must not create duplicate charges or credits. We built recovery flows and idempotent payment and generation handling to make retries predictable.

Accomplishments that we're proud of

We connected image understanding, editable 3D motion, AI video generation, and a payment-backed credit system in one project. Creators can review the exact motion clip before submitting a render and compare that motion with the generated result afterward.

We also made the editor embeddable and extracted the motion timeline into a reusable package, giving the work value beyond this application.

What we learned

Simple geometry can be a useful creative interface for AI. A rough blockout gives people a concrete way to express spatial intent and timing without modeling a detailed scene.

We also learned that an AI workflow needs more than a successful model call. Clear previews, visible costs, saved state, and reliable recovery all contribute to whether creators can trust the tool.

What's next for moblocks

We want to improve video object tracking, add manual object relinking and moving-camera support, and make render progress easier to follow across the editor. Before opening public access, we also plan to complete end-to-end PayPal sandbox purchase and webhook validation, then refine the experience through creator feedback.

Built With

  • ai3d
  • fal
  • h3max
Share this project:

Updates

Submission history