TypeCast

Inspiration

A script says "INT. 1980s POLICE STATION, NIGHT." Somebody now has to find that room. It has to look right on camera, be deep enough for the lens, have space nearby for three trucks and thirty crew cars, and belong to a building that could actually be rented. Today that means hours of searching listings, scrolling photos, and driving to places that turn out to be wrong.

Student crews and small productions feel this most. They can't afford a location team, and when a found location doesn't work, they can't afford to build a set or pay for CGI either. We wanted to give them the first pass of a location scout in minutes, plus a way to see what a real room could become before anyone paints a wall. We also wanted it to be honest about what it knows and what still needs a site visit (a recce).

What it does

Type a scene. Typecast finds real places that can play it, and then helps you imagine what they could become.

Real interiors. AI agents search 10,568 camera views from Matterport 3D scans of 343 CoStar properties in Los Angeles and Houston. They return three ranked buildings with real frames from inside each one.
Physical checks. For each frame, Typecast checks whether the shot physically fits the room, given the lens and the room's depth and ceiling. It estimates crew parking from Vexcel aerial imagery and flags nearby freeways, rail lines and airports.
A skeptic. A separate agent argues against the shortlist with source-backed objections, such as "Parking short: 1 truck against 3 needed." An attractive room that the crew can't work in gets called out.
Follow-ups. Say "need parking for 5 trucks" and the same conversation re-ranks the candidates it already found, without searching again.
Look around. A 360° viewfinder lets you compose the shot inside the scan and switch lenses from 24 to 50 mm.
Reimagine. An Edit (AI) chat restyles the real room view: "dress it as a 1970s precinct", then "night, desk lamps". The walls, doors and roof trusses stay where they are. A wipe compares the real scan with the imagined set, and every AI image is labelled "AI-imagined set dressing, not the real room."
Exteriors and outdoor places. Vexcel oblique search finds rooftops, alleys and hillsides, shows them from real 45° angles and in a 3D city view, and reports terrain slope and relief. A building card shows roof type, height, skylights and defects. You can ask questions about a place and get answers drawn from live Vexcel data, then reimagine the exterior ("rooftop bar at night").
Director's Room. Give it a whole script and it groups scenes by building to reduce company moves, the times the whole crew packs up and relocates.
Inquiry drafts. Approving a pick saves a local inquiry draft for the owner. Nothing is ever sent automatically.

How we built it

Layer	What it decides
CoStar	Which property it is, its type and era, and whether it has marketed lease availability
Matterport	What the camera can see: real scan views for retrieval, ranking, the viewfinder and reimagining
Vexcel	Where the crew can stage: aerials, parking masks, oblique exteriors, 3D cities, terrain, building structure
OpenStreetMap	Mapped parking, walking routes, and distance to noise sources (an estimate, not a sound measurement)

Agents. LangChain deepagents runs a director that writes a plan. It hands work to market, interior and logistics scouts, with the interior and logistics scouts running at the same time, and then to a read-only skeptic. Draft approval goes through a real human-approval checkpoint. A simpler fixed-order engine uses the same tools and serves as a fallback. Progress updates in the UI are streamed from the real tool calls, not replayed, and runs are traced in LangSmith.

Retrieval. CLIP embeddings over cut perspective views from each scan's panoramas, then a vision model that re-ranks the top candidates against the exact scene text.

Shot fit. For a lens with focal length $f$, a sensor of width $s = 24.9\,\text{mm}$ and a frame of width $W$, the camera has to stand

$$d_{\text{camera}} = \frac{W \cdot f}{s}$$

from the actors. Adding a fixed margin for space behind the actors and for the operator gives the room depth the shot needs. A 24 mm lens framing 6 m needs about 8.78 m. If the room's depth or ceiling is unknown, the result is unknown, never fits.

Reimagining. Edits go through a switchable image model (Gemini or OpenAI) that keeps the parts of the photo it wasn't asked to change. On a real oblique, the fastest model finished a full edit in about 4 seconds. Separately, World Labs Atlas uses depth derived from the scan to keep AI set dressing consistent with the room's geometry.

Stack. Python and FastAPI for the backend. Plain JavaScript modules with Leaflet and three.js for the viewer, with no build step so it runs offline at the venue. Licensed data stays in a private cache.

Challenges we ran into

Keeping images honest. It's easy to make a beautiful picture that implies things nobody checked. We had to keep "real scan", "estimated from aerial" and "AI-imagined" visibly separate throughout, including during the wipe animation.
Unknown is not zero. Missing parking, depth or ceiling data had to stay unknown end to end. That meant no zero values, no "fits" based on guesses, and no straight line between two points passed off as a walking route.
Linking three datasets. Joining CoStar properties, Matterport scans and Vexcel imagery to the same building required exact source mappings rather than address guesses. Parking used real segmentation-mask pixels, holes included, instead of rough bounding boxes.
Speed. The multi-agent engine started at about 84 s per run. Running the scouts in parallel, sending the model short summaries instead of full records, and lowering the scouts' reasoning effort brought the warm median down to about 41 s (about 9 s for the fixed engine).
Measuring retrieval fairly. We froze the index and candidate pools before human review and tuned only on separate tuning scenes, so we couldn't grade our own homework.

Accomplishments that we're proud of

A creative loop on top of real evidence: find a real room, compose the shot, then reimagine it as the set you need, with the real architecture kept and the AI clearly labelled.
A working end-to-end flow on real data: scene → agent plan → real interior frames → parking → skeptic → follow-up → reimagine.
73.33% precision@10 (lower bound) on 15 held-out scenes judged by a person, up from 66% before our changes. That's one reviewer, and the second number is a repeat on the same scenes, so we report it as a measurement rather than a general accuracy claim.
All five rehearsed demo scenes passed on both engines, including the follow-up re-rank and the unsent drafts.
Outdoor scouting built on Vexcel's own MCP server, with read-only tools and a cap on how much each question can query.

What we learned

A useful scout gives you reasons to visit and reasons to hold off. The skeptic turned out to be one of the most valuable parts because it shows where the evidence runs out.

We also learned to keep three things separate: retrieval quality, live latency, and how good the demo looks. The visual reveal makes the sources easy to understand, but it doesn't change what the benchmark proves.

What's next for TypeCast

Cost and travel with real quotes: the drive-time and budget filters and the location vs. standing set vs. build vs. CGI comparison are live, using public price ranges. Next, swap those ranges for real vendor quotes.
More cities, more scanned buildings, and wider human evaluation with more than one reviewer.
Interiors you can walk through, generated from sparse scans.
Validation with working location managers before we claim time saved.
Free access for student crews, and a way for owners of unused buildings to list them for filming. These are goals, not results yet.