Pedestrian Realm 311

Inspiration Cities are drowning in 311 requests. A cracked curb ramp gets reported, and a human dispatcher has to manually look up ADA regulations, call the utility company to check for buried gas lines, and figure out which of four departments handles it. Meanwhile the ramp stays broken, someone in a wheelchair can't get through, and the SLA clock is ticking. We wanted to see if an AI agent could handle that entire triage chain in seconds.

What it does Pedestrian Realm 311 is a conflict-aware infrastructure triage agent built on Elastic Agent Builder. When a citizen reports a sidewalk, curb ramp, or crosswalk issue, the agent:

Finds nearby infrastructure within 200 meters using geo-distance queries Detects ADA regulation violations by comparing slope measurements against federal standards (Section 406: max 8.3%, Section 402: max 5.0%) Identifies underground utility constraints like gas lines and water mains that require special coordination Routes the work order to the correct department automatically Monitors all open work orders for SLA breaches and flags overdue items Updates the city's global compliance score when repairs are completed How we built it The entire domain lives in a single Elasticsearch index called pedestrian_realm using the ORCA data model — Objects, Regulations, Constraints, and Actions — with an object_type discriminator so regulations, infrastructure, utilities, departments, service requests, and work orders all coexist in one index. We used ES|QL for geo-distance infrastructure search, ADA violation detection, constraint analysis, and SLA breach monitoring with date math. Elastic Agent Builder ties it all together with a system prompt, built-in ES|QL execution tools, and three YAML workflows that automate the full triage-to-completion lifecycle. A Next.js app at pedestrian-realm.vercel.app provides a live demo interface.

Challenges we ran into Modeling 7 different object types in a single index without field collisions required careful mapping design. SLA breach detection using date math across time zones was trickier than expected. The routing logic also needed precise CASE expressions to handle every triage branch correctly — violations with constraints, violations without, crosswalk-only issues, and general maintenance all route differently.

Accomplishments that we're proud of The single-index ORCA pattern is something we're really proud of — it keeps the entire municipal domain queryable in one place and makes ES|QL joins across object types clean and fast. Watching the agent detect a specific ADA regulation violation, cross-reference it against a utility constraint, and route to the right department — all in one reasoning chain — felt genuinely powerful.

What we learned Elastic Agent Builder is remarkably capable when the data model is well-designed. The combination of geo-distance search, regulation cross-referencing, and constraint detection in a single ES|QL workflow creates intelligence that would be difficult to replicate with traditional rule-based systems. We also learned that platform.core.execute_esql and platform.core.generate_esql are incredibly powerful built-in tools that let the agent reason about data in ways we hadn't anticipated.

What's next for Pedestrian Realm 311 Real-time integration with city GIS systems for live infrastructure data, expanded coverage beyond pedestrian infrastructure to include traffic signals and stormwater assets, predictive SLA modeling based on department workload, and a citizen-facing portal where residents can track the status of their 311 reports in real time.

Built With

• claude
• elastic-agent-builder
• elasticsearch
• es|ql
• next.js
• node.js
• react
• tailwind-css
• typescript
• vercel