parcel

interview mode
freestyle mode

inspiration

site planning is the first 5 minutes of any housing project. somehow it costs $3,000/year in Autodesk Forma or TestFit licenses. small developers, ADU owners, community land trusts, and affordable-housing nonprofits get priced out of step zero. parcel turns that step into a sentence.

parcel reads setbacks, lot lines, and zoning so anyone can draft a 3D site plan without a CAD seat.

what it does

input	parcel does	output
typed prompt	Gemini agent loop with constraint validation	3D site plan in browser
voice interview (5 questions)	guided by ElevenLabs TTS + Web Speech API STT	same
freestyle voice convo	Gemini chat seam with canned fallback	same

after the shell is placed, parcel layers:

interior floorplan + furniture (separate Gemini route per building)
trees, walkways, fences, bushes, pools, parking, street props
on-chain provenance via Solana mint (Metaplex mpl-core + Irys uploader)
saved past plans, persisted with backboard-sdk

how we built it

1. the JSON contract

every tool, the agent, the Zustand store, and the R3F scene speak one type from lib/types.ts:

// origin = front-left of lot. +x = right (width). +z = back (depth). feet.
export type SitePlan = {
  lot:       { width: number; depth: number };
  setbacks:  { front: number; back: number; side: number };
  buildings: Building[];     // x, z, w, d, stories, material, structure_type, program
  parking?:  { x: number; z: number }[];
  trees?:    Tree[];         // species ∈ {oak, pine, palm, maple}
  walkways?: Walkway[];      // flagstone | concrete | asphalt
  fences?:   Fence[];        // wood | wrought-iron | hedge
  bushes?:   Bush[];         // boxwood | hedge_round | flowering
  pools?:    Pool[];         // rectangle | round | kidney
  props?:    StreetProp[];
  status:    'pending' | 'valid' | 'invalid';
};
export type Step = { tool: string; note: string; ok: boolean };

2. the agent loop (`app/api/plan/route.ts`)

runtime: Node.js on Vercel Fluid Compute, maxDuration = 60s
model: gemini-3-flash-preview via @google/genai
calling mode: FunctionCallingConfigMode.ANY + ThinkingLevel.LOW
iteration cap: MAX_ITERATIONS = 20 (fresh draft) / MAX_MODIFY_ITERATIONS = 8 (edit existing plan)
rate limit: in-memory token bucket per IP (lib/rateLimit.ts)
memory: loads last 5 prompts + plan summaries from Backboard so vague follow-ups inherit defaults

loop:
  1. send prompt + plan-state directive + tool declarations to Gemini
  2. read functionCalls[] off the response
  3. for each call: TOOLS[name](plan, args) → { plan, note, ok }
  4. push a Step into the trail; feed function-response Parts back
  5. break when finalize fires or iterations exhausted

the model never emits 3D geometry. it only fills typed JSON. our code does the math.

3. the 11 tools (`lib/tools.ts`, ~1.3k lines)

tool	what our code does
`set_lot`	writes `lot` + `setbacks`, computes buildable envelope
`place_building`	rectangle-inside-rectangle setback check via `isInsideSetbacks`, no overlap with existing buildings
`check_setbacks`	validator Gemini calls before committing — feeds error string back so model retries
`place_parking`	greedy 9×18 ft stall packing in remaining space, respects driveway clearance
`place_trees`	per-species canopy radius, avoids buildings + walkways + each other
`place_walkway`	start/end + width + material, snaps to nearest building entry
`place_fence`	side flags (front/back/left/right) + style
`place_bushes`	canopy-aware avoidance, default diameters per variety
`place_pool`	shape + dims, validates clearance from buildings
`place_street_furniture`	benches, lamps, planters at front-yard zones
`finalize`	flips `status` to `valid`, ends loop

key insight: the validator is a tool, not a post-step. when place_building returns ok: false, Gemini sees the failure note in the next turn and adjusts. constraint reasoning, not generation.

4. the renderer

React Three Fiber 9 + drei (<Grid>, <OrbitControls>, <Box>, <Gltf>)
Zustand store that the scene subscribes to per-slice
procedural geometry by default; GLTF models (Quaternius City pack) layered in via lib/modelConfig.ts per tree/vehicle/prop kind
story-stacked buildings (STORY_HEIGHT_FT = 10), dashed setback lines, drag-to-rotate camera
LCP fix: the canvas is dynamic-imported off the landing-page critical path so first paint ships fast

5. voice

layer	tech
transcription	browser Web Speech API (Chrome/Edge)
TTS	ElevenLabs via `/api/tts` proxy (key server-side, rate-limited per char)
voice-mode prompts	Gemini in interview mode (5 structured questions) or freestyle chat
chat fallback	`gemini-2.5-flash` if `gemini-3-flash-preview` returns 5xx

6. on-chain mint (`/api/mint-plan`)

canvas snapshot (PNG) → Irys uploader (Arweave) → Metaplex Umi
                      → mplCore createV1 → Solana asset

every minted parcel becomes a verifiable record the user owns.

7. routes

/api/plan          agent loop (text → site plan)
/api/chat          freestyle conversation
/api/chat-history  Backboard-backed transcript persistence
/api/floorplan     Gemini → room layout per building
/api/interior      Gemini → furniture per room
/api/mint-plan     Solana mint via Metaplex
/api/save-memory   Backboard preference write
/api/tts           ElevenLabs proxy

stack

Next.js 16 · React 19 · TypeScript · Tailwind v4 · Three.js 0.184 · @react-three/fiber 9.6 · @react-three/drei 10.7 · framer-motion 12 · zustand 5 · @google/genai 2.0 (gemini-3-flash-preview, gemini-2.5-flash fallback) · ElevenLabs · @metaplex-foundation/mpl-core + umi-uploader-irys · bs58 · backboard-sdk · Vercel

challenges we ran into

agent looping forever. early runs got stuck in place_building → check_setbacks → place_building cycles. fix: hard cap at 20 iterations, separate cap of 8 for modify mode, and a MODIFY_DIRECTIVE system prompt that disables set_lot so edits can't restart from scratch.
shared type drift. with 4 people writing tools, store, scene, and API in parallel, divergent field names would have killed integration. one lib/types.ts, no exceptions.
landing-page LCP. the 3D engine was eating first paint. moved <AccentGrainient /> and the heavy R3F canvas behind next/dynamic with ssr: false.
voice race conditions. Web Speech emits interim/final transcripts at unpredictable cadences while ElevenLabs streams audio. needed a careful state machine in lib/speech.ts so the user can't talk over the agent.
Solana on the clock. Umi keypair loading from base58, Irys funding edge cases, mpl-core asset creation. hardened /api/mint-plan and added a fallback canvas-capture path so partial failures still mint something.
Gemini JSON shape volatility. gemini-3-flash-preview occasionally returns malformed function-call args. wrapped every tool entrypoint in runtime checks that surface a clean error string back into the loop.

accomplishments that we're proud of

a real multi-tool constraint-reasoning agent, not a wrapped chatbot
11 typed tools with a validator-in-the-loop pattern that lets Gemini self-correct
3 input modes (text, interview, freestyle) sharing one agent and one JSON contract
on-chain provenance for civic site plans — your design becomes a permanent attributable record
fast landing page despite shipping a full 3D engine
it actually works on real prompts: infill duplexes, mixed-use 4-stories, corner lots with shared driveways, fire stations, public libraries

what we learned

LLMs are far better at constraint reasoning than at generation when the validator lives inside the tool surface. a failing place_building that returns "x+w=85 exceeds buildable max=80, retry" produces smarter output than any prompt phrasing
typed JSON contracts > prose contracts. every line of SitePlan was load-bearing — adding program?: string to Building unlocked the entire interior pipeline without touching the exterior agent
Fluid Compute > Edge for 60s agent loops with multiple round-trips
dynamic-import the heavy stuff. SSR-disabled imports for R3F and WebGL backgrounds bought us a sub-second LCP on the marketing surface
rate-limit the things that bill per character before you ship the proxy

what's next for parcel

real zoning-code lookup by address (San Francisco Planning Code + Santa Cruz code parsers)
multi-building campus planning with shared circulation
export to IFC / DXF so plans flow into traditional CAD pipelines
collaborative parcels — share a link, edit the JSON together, agent assists both seats
partner pilot with an affordable-housing nonprofit to validate the workflow against real ADU projects