pax

A physical-world model of Manhattan you explore through a conversational AI: comfort-routing orchestration and inspection modules that make moving through the city cooler and calmer.

Comment

Inspiration

Walking Manhattan in summer is a thermal and crowding problem that maps ignore. Routing apps optimize for distance or time, and none of them say the block ahead sits in full sun at 3 PM or runs packed with pedestrians. pax started from one idea: a map you can talk to, a model of the physical city that a conversational AI can open and explain, so a walk gets planned for comfort instead of speed.

What it does

pax is a physical-world model of Manhattan you explore through a conversational AI host. It renders the city in 3D with shade, radiant heat (MRT/UTCI), and street-camera pedestrian crowd stress. Pick two landmarks and a time of day, and pax compares five routing strategies to find a cooler, calmer walk to the same destination. The managed-agent host opens modules to inspect any block or route and explains what it finds with inline citations. Routes worth keeping save to a shared "coolest routes" feed.

How it is built

pax runs on Next.js on Vercel for the frontend and API. Amazon DynamoDB is the primary backend, using a single-table design with a GSI to store saved routes, the community feed, field-study sessions, observations, and decision audit records. The comfort engine runs on AWS Lambda and returns five routing strategies with comfort and stress-reduction tradeoffs. The conversational host uses managed agents in a prepare-then-narrate loop over live routing and inspection tools, calling models through Vercel AI Gateway to Amazon Bedrock, with every reply grounded in a tool result. 3D tiles stream from Amazon CloudFront and S3. Sign-in uses WorkOS AuthKit.

Challenges we ran into

Keeping the AI host honest was the hardest part. It has to explain the model without inventing numbers, so the build used a truth-labeled, citation-backed narration path in place of free-form generation. Lambda cold starts of about 80 seconds made live demos fragile until a warm-up path fixed it. Rendering a full 3D city model at interactive frame rates took a dedicated performance pass over shadow budget, terrain, and draw calls. The sun and shade state was static at first, so making the time-of-day slider move the sun and recast shadows became a focused fix.

Accomplishments

pax is a real, live physical-world model of Manhattan running publicly with no login required. The conversational host cites every claim it makes, so a user can check what the model says. Comfort routing surfaces measurable shade gain and stress reduction against the shortest path.

What was learned

A framing from human-AI interaction research shaped the product. Interfaces expose a thin set of choices while the real scene of sun, heat, and crowding across the day stays hidden, so the value comes from surfacing that hidden scene and reframing the AI's signals from "looks clear" to "is adequate for this block." Honest data discipline mattered too: modeled values stay labeled as modeled, and the product never implies measured data it does not have.

What's next for pax

An embedding and retrieval layer (vector store and per-user memory) so the host remembers intents across sessions and fuses the thermal scene, history, and points of interest into one explanation. This is designed this cycle and not yet built. A measured weather source will complement the modeled comfort layers. Shade-window routing will pick the time as well as the path, with export to acoustic and AR interfaces and a mobile app to follow.

Built With

Share this project:

Updates