Waypoint

Image of the Explorer View
Image of the 3D Converter
Image of the Map View
Image of the start menu

Inspiration

We live in areas where the surroundings keep changing. When a new building is proposed, most people only see a flat rendering and have to imagine how tall it is, how it sits on the street and what it replaces. We wanted a way to see a proposal in place, at street level, before anything is built. And we wanted anyone to be able to do it, even without a 3D artist, to take a photo of a building and drop it straight into the map.

What it does

Waypoint is a 3D explorer of the world. Pick any spot on the map and a radius of 100, 200 or 300 m. The real buildings, roads, trees and terrain of that area appear in 3D, and you can walk or fly through them.

Swap buildings: Select a building, delete it, and replace it with your own 3D model, which fits itself to the original footprint. Change your mind? One click on a button puts everything back. Image → 3D: Upload a photo of a building and Waypoint turns it into a 3D model in a few seconds. Set its real-world width, depth and height, then send it to the explorer and place it on the map. Edit history: Every change goes into a log. Jump the camera to any change, revert the scene to that point, or hit Replay and watch the whole design fly past in order. Shared with your team: Edits and imported models are stored in Tiger Data, so a teammate opening the same area sees the same buildings and your edits appear on their screen live. Restyle with AI: Gemini describes nearby buildings (materials, colours, windows, roof shape) and the scene restyles itself to look more like real photos and Gemini data.

How we built it

3D scene: Three.js. Buildings are extruded from OpenStreetMap footprints, with pitched roofs, real 3D windows, doors and trim. Map data: the Overpass API for buildings, roads, parks and trees. Mapbox provides the map and the elevation data for the terrain. Image → 3D on Baseten: we packaged TripoSR with Truss and deployed it on a Baseten GPU (an L4). Our Express server forwards the photo to the model and hands back a GLB. The API key stays on the server and never reaches the browser. Tiger Data: a Node/Express server stores everything in Postgres/TimescaleDB. The edit log is a time-series table, and imported .glb files are stored in the database and de-duplicated by hash, forming a shared model library. Live sync: each browser polls for edits stored since its last cursor and applies the deltas, so teammates' changes appear without a refresh and Undo never wipes out someone else's edit. Custom model: we created several 3D models of buildings as a GLB file, to show the swap.

Challenges we ran into

Getting a GPU model to build: TripoSR depends on a library (torchmcubes) that would not compile on Baseten. We replaced its marching-cubes step with a scikit-image version, so the model runs unchanged. Raw model output: The generated meshes have no material or normals, so they first rendered as black, faceted slabs. We fixed the material and lighting, and added width, depth and height controls for models that come out flat. Sharing more than edits: An edit that places a model is useless to a teammate who does not have the file. We built the shared library so placed models download automatically. Merging edits: We had to keep undo, revert and replay working while other people's edits arrive at the same time.

Accomplishments that we're proud of

A full pipeline from photo to a 3D building standing in a real street, without leaving the app. A shared, live edit history backed by a real database rather than a local file. Every edit can be undone, reverted or replayed.

What we learned

How to package and deploy a machine-learning model with Truss, how to model an edit history as time-series data, and that image-to-3D works best with a real photo of one building from a slight angle, not clip-art.

What's next for Waypoint

Expand beyond Kitchener–Waterloo to any city. Better image-to-3D from several photos of the same building, and cleaner geometry. Shareable links, so a proposal can be sent to someone as a single URL. A bigger area than 300 m by streaming buildings in as you move.