Cities in the Past World Model

Inspiration

Historical photographs show us what a city looked like, but leave us wondering what was just outside the frame. What would it feel like to turn the corner, walk down the street, or explore the neighborhood?

We built Cities in the Past World Model around that curiosity: turning archival photographs from something you look at into a starting point for somewhere you can explore.

What it does

Choose a city and a decade, and the app searches for historical photographs to anchor an interactive, AI-generated world. You can explore the resulting scene using WASD and mouse-look controls.

The experience brings together:

• Archival photo discovery, ranking, and image preparation. • Decade-specific prompts that guide the generated environment. • Photo credits and licensing information. • Re-seeding controls to restart exploration from another selected photograph.

When suitable photographs are unavailable, the app reports that limitation rather than forcing a result.

This is an AI-generated interpretation grounded in historical imagery, not an exact reconstruction of the streets or events of the past.

How we built it

We built a full-stack Next.js and TypeScript application, keeping the interface and backend orchestration in one codebase.

The pipeline geocodes the selected city, searches Wikimedia Commons, Europeana, and Flickr Commons, and ranks candidate photographs for relevance and suitability. Google Custom Search provides a fallback when archive coverage is limited.

Selected images are prepared with sharp, normalized to a landscape format, and stored in Vercel Blob. Upstash Redis provides caching and rate limiting.

A decade-specific prompt and the prepared photograph then seed a Reactor world-model session. We implemented a shared adapter interface for LingBot World 2 and Happy Oyster Adventure, allowing model-specific behavior without tying the interface to one engine.

We also designed an optional Modal / Real-ESRGAN restoration path, with a CPU-only fallback. Browser access uses short-lived, model-scoped tokens; API keys stay on the server.

Challenges we ran into

Finding the right photograph was harder than finding any photograph. A search result might match the city and decade but show a document, a close-up, or a viewpoint unsuitable for exploration. Image quality, composition, provenance, and licensing all mattered.

Interactive generation required careful session management. Movement controls, streamed video, reconnects, re-seeding, and cleanup had to work together— especially because live sessions are metered.

Different models behaved differently. Image requirements, control vocabularies, and startup behavior needed model-specific handling. Mock tests helped us develop safely, but live testing remained essential.

We also had to account for generated-world drift: starting from a real photograph does not guarantee that everything generated beyond it remains historically or spatially accurate.

Accomplishments that we're proud of

• Achieving a live LingBot smoke test with advancing video and working WASD input. • Connecting archival sourcing, image preparation, prompt composition, and interactive exploration in one application. • Establishing a visually reviewed Amsterdam, 1960s reference set. • Building unit and browser regression coverage for controls, navigation, credits, cancellation, and session behavior. • Keeping source attribution, honest limitations, and model flexibility central to the design.

What we learned

A convincing historical experience starts before generation. Choosing a useful camera perspective, checking dates and licenses, and preparing the image carefully are core product decisions—not just preprocessing.

We also learned to distinguish software correctness from model quality. A passing integration test cannot tell us whether a generated street feels coherent or historically plausible. Automated testing and visual review serve different purposes, and both are necessary.

What's next for Cities in the Past World Model

Our immediate priorities are completing Happy Oyster live-video validation, finishing model calibration, and expanding the reviewed city-and-decade collection.

Next, we want to improve spatial consistency through historical maps and GIS alignment, and investigate persistent, revisitable environments using 3D reconstruction techniques such as Gaussian splatting.

We also envision location-aware historical narration that adds context to exploration while clearly distinguishing documented facts from generated content.

The goal is to make urban history more accessible and engaging—without losing sight of the evidence that makes it meaningful.