Groundwork SF

Turn fragmented civic records into an inspectable context graph for community housing teams, with every claim tied to a source and every stale, missing, or proximity-only signal kept visible.

Comment

This document preserves the public project copy and demo script. It is a submission artifact, not the operational source of truth; current implementation and deployment status remain owned by the repository README and the deployed application's health response.

Project overview

Tagline: Every public-record claim should carry its proof.

Groundwork SF combines a proof-carrying civic evidence graph with a Function-backed DigitalOcean AI agent so community housing teams can inspect every claim, source, and limitation around a site.

Inspiration

Community land trusts and affordable-housing organizations often need a quick, defensible picture of what is happening at a site. The relevant public records exist, but they are spread across parcel, assessment, permit, development, housing-program, hazard, and neighborhood datasets. A polished summary can make that fragmentation worse by hiding source dates, missing rows, and conflicting records.

We built Groundwork SF around a simple belief: every civic-data claim should carry its proof.

What it does

Groundwork SF is a proof-carrying site-context explorer for San Francisco community housing research. Choose one of three demo sites and the interface synchronizes:

  • a parcel map;
  • a navigable context graph;
  • focus views for housing, permits, hazards, and neighborhood signals;
  • evidence records with dataset identity, record key, observation date, source date, license, and official source link; and
  • deterministic trust metrics for citation coverage, freshness warnings, conflicts, coverage gaps, and proximity-only evidence.

Clicking a graph claim opens the exact evidence record that supports it. A missing row stays a coverage gap instead of becoming a negative claim, and a nearby 311 count stays a neighborhood signal instead of being attributed to the parcel.

The deployed release compiles three featured sites from bounded live DataSF queries across seven public datasets. Every live graph carries source and retrieval dates, diagnostics, and content-addressed evidence artifacts. The API reports live, stale, refreshing, and fixture states; the interface labels fixture fallback and keeps source-level freshness diagnostics visible.

The embedded AI assistant provides a conversational path through the same evidence. Before answering any site-specific question, the DigitalOcean agent calls a secure Function to retrieve a bounded graph packet. Its answer cites only URLs from that packet, repeats material freshness and coverage warnings, and refuses legal, safety, valuation, suitability, ranking, or investment conclusions. The graph remains the source of truth; the model explains it.

How we built it

One canonical graph contract owns sites, entities, assertions, evidence, diagnostics, and release metadata. A bounded DataSF compiler creates validated live snapshots; Managed PostgreSQL stores current graph state and private DigitalOcean Spaces stores immutable source projections. A SHA-256 manifest and Pydantic models validate the deterministic fallback release.

A FastAPI service owns four public read routes plus a protected agent-context route. The React and TypeScript frontend consumes those routes through one ContextClient boundary. Cytoscape.js renders the graph, MapLibre GL JS renders the map, and URL-backed selection makes sites, focus views, and evidence records deep-linkable.

The product runs as one multi-stage Docker image on DigitalOcean App Platform. Its health response exposes the deployed Git revision and graph release so the demo can be tied to the exact code and data that were tested.

For the AI path, a secure DigitalOcean Function validates scalar tool inputs, calls the protected application route with a separate bearer credential, rejects redirects and oversized responses, and verifies the returned packet's SHA-256 digest. A glm-5.2 agent on DigitalOcean Gradient AI Platform is instructed to call that Function before every site-specific answer. The browser receives only the generated widget's public identifiers; no model key or Function credential reaches the client.

A methodology-only DigitalOcean Knowledge Base backed by Managed OpenSearch provides stable graph semantics without owning site facts. The replacement agent remains private until its fixed 50-query managed evaluation passes; the deployed widget stays on the prior Function-backed agent until that gate clears.

We verified the deployed flow end to end: the agent produced a real Function trace for factual and uncertainty questions, preserved packet status, dates, diagnostics, and citations, and refused an investment recommendation. The frontend uses DigitalOcean's generated streaming widget, so the provider owns chat transport, history, feedback, and rendering while Groundwork owns the evidence product.

We test the Python boundaries with pytest and unittest, the frontend with Vitest, and the desktop/mobile product flow with Playwright and axe. The same browser suite runs in deterministic mock mode and against the deployed API contract.

Architecture

Groundwork uses DigitalOcean for the complete deployed path while keeping one deterministic graph contract at the center.

                              DIGITALOCEAN

 Browser                App Platform
 React + MapLibre  <-->  React/FastAPI container
 + Cytoscape                    |
      |                         +--> Managed PostgreSQL 17
      |                         |    snapshots + refresh leases
      |                         +--> private Spaces
      |                         |    content-addressed DataSF artifacts
      |                         +--> seven bounded DataSF queries
      |
      +--> generated widget --> GLM-5.2 Agent
                                     |
                         site facts  +--> secure Function
                                     |       |
                                     |       +--> protected graph packet
                                     |
                         methodology +--> Knowledge Base
                                             |
                                             +--> Managed OpenSearch

Live evidence plane

For each featured parcel, the FastAPI service first reads the current PostgreSQL snapshot. A fresh snapshot returns immediately. A stale or missing snapshot can acquire a fenced refresh lease, fetch seven fixed DataSF projections, compile a typed graph, upload canonical source JSON to private Spaces, and atomically publish the graph and evidence rows. A failed refresh serves a clearly labeled stale snapshot or the verified fixture fallback; it never converts missing data into a confident claim.

The graph itself is a typed application contract:

Site -> Entity -> Assertion -> entity or literal
                    |
                    +--> EvidenceRecord -> official URLs + dates + license + SHA-256
                    |
                    +--> Diagnostic -> freshness | conflict | gap | proximity-only

PostgreSQL is the durable snapshot store, not the semantic owner. Deterministic compiler code owns graph meaning, and Spaces preserves the exact bounded inputs that produced each evidence record.

AI and graph-aware RAG plane

Every site-specific question must call the DigitalOcean Function. The Function validates scalar inputs, makes one authenticated request to the protected App route, bounds the response, and verifies the graph packet digest before the Agent sees it. The Agent may cite only packet URLs and must preserve packet status, dates, conflicts, gaps, and proximity limits.

Stable methodology follows a separate RAG path. A private replacement Agent retrieves graph semantics and responsible-use policy from a Spaces-backed Knowledge Base indexed by Managed OpenSearch. That Knowledge Base contains no featured-site facts and cannot replace a failed Function lookup. The deployed public Agent remains Function-backed while the replacement stays private until its fixed 50-query evaluation passes.

Failure boundaries

  • LIVE_DATA_ENABLED=false rolls the API back to the verified fixture release.
  • A failed or competing refresh preserves a stale snapshot instead of risking an incomplete write.
  • A missing or invalid Function packet produces no site-fact answer.
  • The browser receives widget identifiers, never a model, Function, database, or Spaces credential.
  • Agent promotion is independent from data deployment and happens only after private probes and managed evaluation pass.

Why DigitalOcean is central

DigitalOcean is the runtime fabric, not a decorative integration:

  • App Platform deploys the single tested frontend/API artifact and exposes its revision through health checks.
  • Managed PostgreSQL and private Spaces separate current graph state from immutable source evidence.
  • Functions provide the narrow, independently secured trust boundary between a public Agent and protected graph data.
  • Agent Platform supplies GLM-5.2 inference and the generated streaming widget, while Knowledge Bases and Managed OpenSearch add isolated methodology RAG.
  • Agent Evaluations make promotion a measured gate rather than a demo-time judgment call.

The full component map, sequences, limits, durability model, and interfaces are documented in the current architecture.

Challenges we faced

The hardest problem was not drawing a graph; it was preserving meaning across boundaries. Dates can describe a source update, an observation, or an event. An absent record may mean a true negative, stale coverage, or a failed join. Nearby events are not parcel facts. We modeled those distinctions explicitly and kept them visible instead of smoothing them into a confident summary.

We also had to keep AI in the right role. Deterministic code owns identity, joins, facts, dates, limits, and citations. The model may explain a bounded packet, but it cannot become the source of site facts. That led us to a small, hash-verified Function contract rather than a general chatbot or natural-language database interface.

Finally, a dense evidence graph had to work on both a 1440-pixel desktop and a 390-pixel phone without hiding provenance or accessibility controls. The final design uses a parallel keyboard list, mobile graph/map/evidence tabs, visible map attribution, deep links, and diagnostics that remain reachable around the chatbot launcher.

What we learned

  • Provenance works best as a product primitive, not a footnote.
  • “Unknown,” “historical,” and “nearby” are different states and deserve different UI and contracts.
  • An AI assistant can be more trustworthy when its retrieval surface is smaller, deterministic, and inspectable.
  • A stable client and repository boundary let us activate live compilation and durable storage without rewriting the experience.

What's next

Next we will expand beyond three deep live sites, improve the compact agent response contract until the fixed evaluation clears its promotion gate, and work with community land trusts to decide which evidence and uncertainty signals are most useful.

Built with

React, TypeScript, Vite, Cytoscape.js, MapLibre GL JS, FastAPI, Python, Pydantic, Docker, OpenStreetMap, DigitalOcean App Platform, DigitalOcean Managed PostgreSQL, Spaces, Managed OpenSearch, Functions, Knowledge Bases, Gradient AI Platform, Agent Evaluations, GLM-5.2, Vitest, Playwright, pytest, and axe-core.

Built With

Share this project:

Updates