LabMind: the autonomous lab assistant

This shows our generated report detailing the time of events, associated tool calls, and reasoning to fix anomalies.
This showcases another anamoly within the report with it's associated time and how the agent understood, analyzed, and fixed the issue.

Inspiration

We talked to research students across Stanford, USC, and UCLA. Three patterns kept coming up: they babysit overnight experiments and lose sleep, they re-run experiments their lab already tried because nobody remembers what happened six months ago, and threshold-based monitoring catches obvious failures but misses the subtle ones that actually matter.

What it does

LabMind is an autonomous lab assistant that runs experiments overnight so researchers don't have to. A researcher uploads an experiment plan to OpenClaw, and LabMind first checks the lab's corpus of past experiments for similar work. It surfaces relevant past failures and what was learned from them. Once the experiment is confirmed, LabMind monitors live sensor readings every 60 seconds, reasons over what's happening using a local Nemotron 3 Super model, and intervenes when needed. It's not just on threshold breaches, but on the subtle patterns that require institutional memory to recognize. In the morning, it hands the researcher a complete report explaining what happened, what it did, and why.

How we built it

Core AI Stack

OpenClaw agent runtime
Nemotron 3 Super running locally via Ollama
NVIDIA DGX Spark for fully on-prem inference

Infrastructure

FastAPI backend for:
- Experiment uploads
- Run management
- CSV logging
- Live analytics ingestion
FastMCP server exposing tools for:
- Sensor readings
- Historical experiment queries
- Intervention logging
- Experiment finalization

Instrument Simulation

Simulated lab instruments running as separate Docker containers:
- Temperature controller
- pH probe
- Microscopy system
Instruments stream readings to the backend at configurable intervals

Deployment

Entire system orchestrated with Docker Compose
No cloud calls
Fully air-gapped architecture

Challenges we ran into

MCP server and OpenClaw agent were both running, but tool calls were not executing
Traced the issue to a transport mismatch
Rebuilt communication using:
- mcp-remote
- MCPorter over SSE

Accomplishments that we're proud of

End-to-End Working Demo

Uploading a YAML intake document from a laptop
Backend automatically creates a run
Agent detects the run and begins monitoring within 60 seconds
Entire workflow runs locally on a DGX Spark

Autonomous Reasoning

Agent correlates multiple instrument signals simultaneously
References historical experiments during interventions
Explains exactly why an action was taken

What we learned

We learned about problems that grad researchers face on a day to day. We also learned how to work with a DJX Spark and how to effectively leverage compute for complex workflows and challenges. We learned how to delegate tasks effectively by user stories for our product.

What's next for LabMind

Full NemoClaw integration: we already wrote our agent's sandbox policies in NemoClaw-compatible YAML; the next step is deploying inside the OpenShell runtime so the agent's filesystem and network access are kernel-enforced which is necessary for any pharma or compliance-sensitive customers.