Tessera

Drop a photo of a place. Tessera finds where the photographer stood, the heading and the lens, then your group plans its own version in a live 3D set, even by text.

Comment

Try it live: https://app.tesseraphoto.com/p/A9TVAN

A film set for the photo you want to take

Drop a photo of a place. Tessera finds where the photographer stood, which way they faced and the lens, then rebuilds the place as a live 3D set your crew plans in. It is for photographers planning a shot, location scouts and film crews, and anyone recreating a photo they love.

Sponsor tracks: a detailed page for each

Every sponsor track has its own page: what we built for it, where it runs in the product, the proof and the limits. All of them: https://tesseraphoto.com/tracks/

Track What we built Strongest proof Page Video
NVIDIA Nemotron runs the set: words to objects, the photo rebuilt as stand-ins, a render that checks its own physics, props at real size, call sheet, voice, offline on CPU; plus our LoRA lens reader 16 objects from one sentence in 6.6 s; 72.4% long-lens recall on unseen landmarks Detailed page Video, 2:38, all 13 uses
SpacetimeDB The whole shared set is one module: every object, job, version and message is a row; no other backend 900 of 900 edits at 10 Hz, median 42 to 50 ms; an outsider sees 0 rows until they join Detailed page Video
Photon The set on the apps people already use: a photo located by text, versions as reply threads, check-ins, props, alerts; a phone call that guides you 9 live round trips on Telegram; the other channels 29 of 29 against a stand-in Detailed page Video
HackUTD Virality Built on what people already post: "where was this photo taken?" threads and recreate-the-photo posts. Every shoot is a share link that opens on a phone, and the set answers in Telegram, so a recreation spreads the way the original photo did 103 of 1,949 Reddit posts asked where a photo was taken from, which lens, or how to recreate it; the top one has 858 points Research page 10 reels
MLH Vultr Production on a Vultr VM in Dallas, Object Storage for photos and props, Serverless Inference for safety, speech and the render Content Safety on Vultr in 0.51 s per text; the app and the site on one VM with automatic HTTPS Detailed page Video

The problem

Google Lens names the place in a photo. It doesn't say where the photographer stood, which way they faced or which lens they used, so people post the photo and ask strangers. We read 1,949 Reddit posts: 103 asked exactly that, and 79 wanted the exact spot. Method and posts

r/AskPhotography: will the sun hit this spot in DUMBO at golden hour?

How it works

  • Find the spot. Our solver matches the photo against our own 3D model of the place, built from 155 Creative Commons photos, and returns where the photographer stood, the heading and the lens. It answers "2.2x on your iPhone", not just a focal length. If it isn't sure, it says "Couldn't place" instead of guessing.
  • Build the set. The place in 3D: Google Photorealistic 3D Tiles for the view, Google Street View through the camera where a panorama exists, and our clay model of the block with your photo projected from where it was taken. Google imagery is display only and never enters a render.
  • Plan the shot. A physical camera with 19 real bodies and 67 lenses, with depth of field, distortion and exposure. Scrub the sun to any hour and the city's light changes with it. Say "two friends walking toward camera" and Nemotron places them in metres through the shot camera.
  • Preview it. An AI render puts your people and props into your own photo. Nemotron checks each object's light, shadow and scale and renders again on a fail. Every render is labelled "AI interpretation".
  • Plan together. Every person, prop, light and camera is a SpacetimeDB row that every screen shares live, so the director's drag shows up in the scout's window. Texts, photo check-ins, renders and golden-hour alerts arrive in the chat through Photon.
  • Go shoot. The call sheet puts golden hour, conditions and the walk between shots on one plan. On the day, the scout screen walks you to the mark: "You're there. Facing the right way. Take it."

Studio's World view: the 3D city, Cam A at its mark, and the photo

Proof

  • Solve: 10 of 11 held-out photos placed, 9 within 25 m of their GPS, median focal error 3%.
  • Words to objects: 16 objects from one sentence in 6.6 s.
  • Live set: 900 of 900 edits at 10 Hz, median round trip 42 to 50 ms.
  • Our own model: a LoRA on Nemotron Nano 12B v2 VL, trained in 44 minutes, catches 72.4% of long-lens photos on landmarks it never saw (the base model: 12.6%).
  • Live texts: 9 round trips through Photon on Telegram.

Through Cam A beside the AI render, with Nemotron's check

How we built it

  • The solver (Python): we built a 3D model of the spot from 155 Creative Commons photos with pycolmap, then match a new photo against it with SuperPoint, LightGlue and MegaLoc to recover the position, heading and focal length. It runs on a CPU.
  • The live set (SpacetimeDB): one TypeScript module is the whole backend. Rows, row-level security, schedules for the sun and alerts, and a procedure that calls Nemotron.
  • The crew service (Node on a Vultr VM): it calls the Nemotron models, the render on Vultr Serverless Inference, Photon and ElevenLabs, behind Caddy with HTTPS.
  • The apps (React, three.js, MapLibre): Studio for the crew, Simple for everyone else, and an Astro site with a page per track.
  • Our model: a LoRA on Nemotron Nano 12B v2 VL, trained on a Brev A100 and served with vLLM.

Challenges

  • FLUX.1-dev returned HTTP 500 for much of the night, so the render moved to Qwen-Image on Vultr, and every result names the model that made it.
  • TRELLIS failed often, so a failed prop becomes a flat cut-out at its real size instead of nothing.
  • Vultr's monthly cap blocked a GPU instance, so the fine-tune ran on Brev.
  • A SpacetimeDB database that other databases have called can't be deleted and recreated, which cost us our dev database once.

Limits

  • One spot is mapped exactly today (DUMBO). A new spot takes about 150 photos and 23 minutes on a CPU. Anywhere else, Nemotron names the place, OpenStreetMap pins it, and Tessera marks it approximate.
  • Renders are AI interpretations, not photos.
  • iMessage, SMS, voice, email and WhatsApp are tested against a stand-in, not live.

Two people on one set, with the live SpacetimeDB feed

What's next

More mapped spots, starting with one we shoot ourselves at UTD; a side-by-side export of the original photo and your recreation; the lens reader on the phone; and the other Photon lines live.

Built With

  • astro
  • brev
  • caddy
  • colmap
  • cuopt
  • elevenlabs
  • fastapi
  • figma
  • google-3d-tiles
  • lightglue
  • maplibre
  • node.js
  • nvidia-nemotron
  • nvidia-nim
  • photon
  • pycolmap
  • python
  • react
  • react-three-fiber
  • riva
  • spacetimedb
  • three.js
  • typescript
  • vllm
  • vultr
Share this project:

Updates

Submission history