Inspiration
Software remembers everything that happens inside it. Physical work remembers almost nothing.
Groundstate is “Git for the Physical World”: a persistent memory agent for laboratory and technical workbenches. A camera observation, spoken note, timestamps, and existing experiment records become claims in a temporal world graph. Every claim keeps its evidence, confidence, validity window, source, and superseded belief.
What it does
The demo follows one small but high-stakes story. Sample A17 is washed and expected to enter the centrifuge in twenty minutes. A later scan finds A17 missing, sample B02 in the expected position, and centrifuge C-01 running. Groundstate produces a semantic reality diff instead of a generic image caption, infers a likely transition, and asks one clarification.
If the operator says “Actually, that was B02,” Groundstate authors a revision, retracts the wrong belief, restores A17’s timer, and updates every dependent assumption. Fast-forwarding the clock wakes the autonomous follow-up and reopens the overdue action.
How it uses Google
- Gemini 3.5 Flash Lite via the Google GenAI SDK receives a camera frame, operator voice note, and remembered world state, then returns a typed observation claim with confidence and contradiction status.
- Cloud Run hosts the production web app and API.
- Firestore stores the temporal world graph: claims, evidence, revisions, and validity windows.
- Pub/Sub receives expected-transition messages so the agent can wake asynchronously when a physical step should have happened.
The hosted browser experience has a deterministic replay for judges, while its live observation path calls the Cloud Run runtime. The same claim contract is populated by Gemini and persisted in Firestore, with follow-up work queued in Pub/Sub.
Collaborative Partner loop
Groundstate is built around the agent leading a human through uncertainty:
- Observe a workbench with a photo and a spoken note.
- Explain the semantic change and show the evidence bundle.
- Ask one focused clarification when identity or state is ambiguous.
- Apply the human correction as a revision across dependent assumptions.
- Wake the operator when an expected physical transition becomes overdue.
What we learned
The hard part is not detecting objects. It is deciding whether the same object across two observations moved, disappeared, changed state, or was replaced—and doing that without silently corrupting a scientific record. High-confidence claims can advance the state machine; ambiguous claims become explicit questions with a visible evidence bundle. A human correction should travel through dependent assumptions instead of overwriting history.
Challenges
Physical reality is partial, asynchronous, and visually inconsistent. We designed a deterministic reducer for the product demo, then kept the cloud runtime boundary explicit so the same claim contract can be populated by Gemini and persisted in Firestore. The UI treats uncertainty as a first-class state rather than hiding it behind a confident paragraph.
Try it
- Open the hosted app and click Scan bench.
- Inspect the A17/B02/C-01 semantic diff.
- Choose Confirm A17 to seal the likely transition, or It’s B02 to see a correction propagate.
- Click Fast-forward 20 min to trigger the autonomous follow-up.
Synthetic lab-workbench entities and experiment records are used: samples A17/B02, reagent PX-9, pipette PIP, centrifuge C-01, and Experiment 28. No personal, medical, or third-party laboratory data is included.
Product boundaries
The hosted app opens in Workspace mode, a clean live surface for the operator’s projects and evidence. Project creation, evidence intake, and the observation API are real product actions. Demo replay is an explicit opt-in fixture containing the synthetic A17/B02 story for a deterministic judge walkthrough; it is isolated from Workspace data.
The familiar project header, tabs, status badges, and activity patterns are intentional interaction grammar. What Groundstate manages is different: camera frames, voice notes, expected physical transitions, semantic reality diffs, human anchors, and evidence-backed revisions—not source files or code branches.
