LucidFrame

Inspiration

A photograph captures one point of view, but our imagination immediately wonders what exists beyond it. I wanted to explore what would happen if an image stopped being something I only looked at and became somewhere I could enter.

LucidFrame began with that idea. Instead of generating another flat image, I wanted technology to transform paintings, photographs, and panoramas into interactive spatial artworks.

What it does

LucidFrame transforms an uploaded image into an explorable 3D Gaussian scene.

Users can upload a photograph, painting, or panorama and choose how it should be reconstructed. Image to 3D creates a detailed scene around the original viewpoint. Image to 360 generates the missing directions before building a complete look-around experience. Panorama to 360 reconstructs an existing panoramic image as a spatial scene.

The result opens inside an interactive browser viewer. Users can rotate the camera, change the field of view, and walk through the scene using keyboard controls. Completed creations are saved in a local library where they can be opened again or deleted.

How I built it

I built the interface with Next.js, React, TypeScript, and Tailwind CSS. The frontend manages image uploads, generation progress, the scene library, settings, and the interactive viewer.

The backend uses FastAPI, Python, PyTorch, and CUDA. It receives the image, runs the reconstruction pipeline, manages GPU memory, streams progress through WebSockets, and compiles the final result into a splat file.

Apple SHARP powers the primary Image to 3D mode. It predicts millions of three-dimensional Gaussians from one image. CubeDiff generates six connected directions for the image to 360. SPAG4D SHARP 360 aligns and merges Gaussian predictions for panoramic scenes. Real ESRGAN can restore smaller or compressed images before reconstruction.

*Three. js * renders the finished Gaussian scenes directly in the browser.

Challenges I ran into

The biggest challenge was depth. A normal image does not contain information about what is behind an object or outside the camera's view. When the camera moves too far, incorrect depth can create gaps, stretched shapes, or unstable details.

Building a complete 360 scene was even harder. The front, back, sides, ceiling, and floor needed to agree on color, orientation, and depth. Small differences between those directions could create visible seams or confusing geometry.

I also had to render millions of Gaussians without making the viewer unusably slow. This required careful GPU memory management, controlled model loading, camera calibration, and more efficient splat sorting in the browser.

Accomplishments that I'm proud of

I am proud that LucidFrame is a complete working experience rather than a collection of disconnected experiments. A user can upload an image, follow its generation progress, enter the finished scene, explore it, and save it in a library.

The primary reconstruction pipeline runs locally on a consumer GPU. It does not require a paid API, and the uploaded image does not need to leave the computer.

I am especially proud of how LucidFrame handles paintings. Brushstrokes, canvas texture, color, and composition become part of the spatial scene. The result feels different from both a traditional painting and a normal 3D model.

What I learned

I learned that adding more pixels or more Gaussian points does not automatically create a better scene. Camera calibration, depth alignment, coordinate systems, opacity, scale, and rendering order all affect whether the result feels stable.

I also learned that uncertainty can become part of the artwork. The original viewpoint comes from the uploaded image. As the visitor moves away from it, the scene gradually becomes the model’s interpretation. LucidFrame makes that transition visible instead of hiding it.

What's next for LucidFrame

I want to improve the consistency and detail of generated 360 scenes, especially in directions that were not visible in the original image. I also want to support several connected photographs so LucidFrame can reconstruct larger spaces with stronger geometry.

Future work will focus on faster generation, smaller scene files, smoother browser performance, and simple ways to share completed scenes with other people.

Built With

• applesharp
• cubediff
• cuda
• diffusers
• fastapi
• gaussiansplatting
• next.js
• python
• pytorch
• react
• realesrgan
• spag4d
• tailwindcss
• three.js
• typescript
• websockets