Relight

Inspiration

Gym progress pictures depend heavily on lighting: from the right angle it shows muscle definition, and from the wrong one it hides it. Most photos share the problem, and you rarely choose the light when you take them. We wanted a fast, free way to move the light afterwards, by dragging it instead of describing it to an AI and hoping.

What it does

Upload a photo, paste one, or drag one in from another web page. Relight works out the photo's shape once: how far away each part is and which way each surface faces. Then you drag a sun around the photo and the highlights and shadows follow, live. You can set the light's height, strength and color, add a second fill light, and export the result. Your photos are kept as drafts on your own device. In the full app, an optional Enhance step sends the result to Gemini with a request such as "golden hour".

How we built it

We have Depth Anything V2 estimate how close each pixel would be to the camera and create a height field for Marigold to add surface direction. Our code makes sure that each pixel is shaded by how it faces the light, and cast shadows are traced across the depth map in near, mid and far layers, so close shadows stay crisp and distant ones soften. It runs in a WebGL2 shader, with a processor version for devices whose graphics chip cannot. The backend is FastAPI and PyTorch, the frontend Next.js. The live demo website needs no server: depth is estimated in your browser. We wrote the code with an AI coding assistant, Claude Code; the three of us directed it and tested everything by hand.

Challenges we ran into

Only measuring depth left the face looking flat, so we added a second model for surface direction.
Shadows fell on the sky as solid blocks, because a depth map treats the background as a wall. We fixed most of those issues with layered shadows.
Flat images, like a sheet of paper or a poster, came out with bands and blotches. The depth model invents a slight bulge, and our shading lights it as real.
In-browser depth came out as a grid of dimples on a Snapdragon laptop and a Samsung phone. The page now checks and tests the graphics chip first and falls back to the processor.
On an iPhone, grabbing the sun at the screen's edge triggered the browser's Back swipe, so we had to change the formatting for iOS browsers.

Accomplishments that we're proud of

I am proud of how accurate the lighting is and of how fast most images are able to load and render.

What we learned

One photo gives a height map, not real 3D, and that sets what is possible. "Works on my machine" means little for graphics code: every new device found a new bug. And direct controls beat prompts when you know what you want.

What's next for Relight

Removing the light already in the photo, sharper surface detail in the browser, and an inbuilt color editor that can help smooth out the smaller artifacts. Also fixing issues with flat images that way they can be processed without the bands and blotches.