REMi 💤
your agent's night shift
💡 Inspiration
At 3 a.m., an alert wakes an engineer. A system is failing, customers are waiting, and the person responsible has to reconstruct what happened across messages, dashboards, and documentation while barely awake.
We started REMi because organizations often protect customer trust by exhausting the people responsible for maintaining it. On-call engineers carry the pressure of keeping services available through repeated overnight interruptions. We wanted to give them support that can investigate, preserve context, and become more useful with experience.
Data centers make this challenge tangible. Uptime Institute’s 2025 analysis reported that nearly 40% of organizations had experienced a major outage caused by human error over the preceding three years. Of those incidents, 85% involved failures to follow procedures or flaws in the procedures themselves. These findings highlight an opportunity for better processes and real-time operational support. Source: Uptime Institute
For us, the social impact starts with the engineer behind the alert. Better support could mean fewer disrupted nights, less time reconstructing routine incidents, and more attention available for difficult decisions. It could also strengthen the services that communities, small businesses, and everyday users depend on.
That led us to REMi: agents that sleep on it and learn from their day.
🤖 What it does
REMi is a local, multi-agent system that works alongside people. Teams create specialized agents, called REMis, assign responsibilities and permitted tools, and collaborate through shared channels and direct messages.
REMi’s workflow extends beyond generating a response. Agents can receive events, gather evidence, execute tools, inspect results, coordinate with other agents, and explain their findings.
🚨 From alert to investigation
Consider an incident-response workflow:
An alert arrives. A local monitoring application reports an incident, such as a GPU crossing a temperature threshold. REMi posts it to the incidents channel and suppresses repeated alerts with the same key and severity within a five-minute window.
The responsible REMi wakes. An incident responder recognizes the event through its configured triggers.
It investigates. The agent runs permitted diagnostic commands inside its Docker sandbox or calls an allowlisted remote tool. Each tool call and result appears in the conversation.
It interprets the evidence. Results return to the agent’s context so it can continue investigating or explain its findings. When human help is needed, it can mention an engineer and describe the next step.
The experience informs future work. Agents save useful notes, while a dedicated Sleep REMi can review conversations and assemble examples for training.
🛠️ Tools with human oversight
REMis can ask one another for help. When a capability is missing, an agent can propose a new tool. REMi tests it in a disposable sandbox and produces a review card containing its code, documentation, and execution result.
A human approves the tool before it becomes available through the shared registry. Each agent’s allowlist determines whether it can use that capability.
🌙 Learning during sleep
Sleep REMi can assemble a dataset, prepare a training specification, and initiate training of a LoRA adapter, a small set of learned adjustments to the base model.
After training, Sleep REMi evaluates the candidate against the base model and can adopt it onto another agent. Adapters are versioned, support weighted combinations, and can be rolled back.
The goal is concrete: when someone asks whether a disk is full, REMi should run a diagnostic check and use the result.
⚙️ How we built it
We combined a Python runtime, a live web console, and locally served NVIDIA Nemotron models.
🧠 Focused context: YAML Agentfiles define each REMi’s purpose, triggers, channels, tools, and budgets. The runtime combines these instructions with personal memory and recent conversation within a default budget of roughly 4,000 tokens.
🔌 Knowledge connectors: Our MCP client discovers remote tools and exposes permitted capabilities to each agent while keeping credentials private. Composio is an intended integration, with real-server validation pending.
🔄 Multi-step execution: Agents receive events, call tools, inspect results, and continue investigating. Timeouts, turn budgets, Stop controls, and repeated-command detection constrain execution.
💬 Shared state and web UI: SpacetimeDB stores conversations, agents, memories, tools, and adapter metadata. A TypeScript module validates operations, while live subscriptions power our phone-friendly React and Vite console.
📦 Local serving and sandboxes: vLLM serves models and adapters. Docker provides persistent agent workspaces and disposable tool-testing containers. GPU access is opt-in and excluded from tool proposal dry runs.
💤 Learning during sleep: PyTorch, Transformers, PEFT, and TRL train local LoRA adapters. The runtime pauses serving on the training node, enforces training limits, and restores serving afterward. Candidate evaluation, versioning, rollback, and Sleep REMi’s protected configuration provide safeguards.
🖥️ Accessible hardware: Our development setup runs on an RTX 5070 Ti with 16 GB of VRAM, creating a path for smaller teams to experiment using hardware they already own.
🧩 Challenges we ran into
Our first challenge was testing the feasibility of the idea itself: could a small local model learn a useful behavior through this workflow?
We had to distinguish learning from memorization. Lower training loss alone could not establish whether an adapter would handle unfamiliar inputs, so held-out evaluation became an essential part of our process. Broader generalization remains an ongoing challenge.
Coordinating multiple machines also introduced complexity. Keeping shared state, agent execution, and model availability aligned required clear boundaries between components.
🏆 Accomplishments that we’re proud of
We are proud of turning an ambitious idea into a working prototype that connects agent collaboration, tool execution, and local model adaptation.
In one recorded development run, a single request led Sleep REMi through:
Training specification → Sleep → Evaluation → Adapter adoption
On a narrow, ten-example held-out command-format task, the adapted model achieved 10/10 exact matches, compared with 0/10 for the base model. This small evaluation is an encouraging demonstration of the learning loop, with broader testing still ahead.
We are also proud of integrating tools into a workflow where people can inspect execution, review proposed capabilities, and retain control over adoption.
📚 What we learned
As an interdisciplinary team, we each learned something new, from specializing language models and smoke testing to managing limited compute and designing agent workflows.
A few lessons shaped the project:
- Evaluation matters as much as training. A promising loss curve needs to be followed by checks on held-out examples and unfamiliar tasks.
- Tool results are part of the reasoning process. An agent needs to interpret what happened after a command, including failures.
- Local compute rewards focus. Small models benefit from concise context, clear tool interfaces, and targeted training examples.
- Autonomy should be earned. Permissions, visible execution, evaluation, and rollback help make increasing responsibility reviewable.
🚀 What’s next for REMi
Over the next week, our priority is to put REMi in front of on-call engineers through small, supervised pilots. Our ambition is to grow toward supporting hundreds of data centers across the country, with evidence from those pilots guiding each expansion.
Next, we want to add voice and phone escalation so REMi can notify the right engineer with a concise briefing:
What triggered the incident? What did REMi check? What did it find? Where is human judgment needed?
Richer connectors could help REMi consult approved runbooks, service histories, and operational tools. As adoption grows, we also want to explore sharing explicitly approved tools and learning artifacts across deployments while respecting each organization’s data boundaries.
We will measure progress through time to a useful diagnosis, unnecessary overnight interruptions, escalation quality, and performance on unfamiliar incidents.
Our vision is a future where reliable services and rested engineers reinforce each other, and tonight’s lessons make tomorrow’s response better. 🌙
Built With
- composio
- cuda
- docker
- docker-compose
- github-actions
- hugging-face-transformers
- json-schema
- lora
- mcp
- nvidia-nemotron
- peft
- pytest
- python
- pytorch
- react
- rest-api
- spacetimedb
- trl
- typescript
- vite
- vllm
- websockets
- yaml