LocalOps AI

Living 3D workspace: Your AI team as animated characters in an interactive 3D office,
Agent workspace: Each agent gets its own page showing measured tasks
Knowledge sources: Upload PDFs, images and documents (read on-device), or link a website or API, then choose which agents can use each one.
Security guardrails: The Sentinel agent enforces fixed tool permissions and blocks unsafe exports

Inspiration

We started with a question: What if you could run your own AI workforce on a computer you control?

Most AI tools give you a chat window. But managing real work requires more than a conversation. Different tasks need different knowledge, permissions, and ways to verify results. We wanted to make those differences visible and understandable.

That became LocalOps AI: a living workspace where specialized AI teammates help with HR, finance, support, and custom tasks, powered by local inference on the ASUS GX10.

Our goal was to combine the usefulness of specialized agents with the privacy of local computing and an interface that feels approachable.

What it does

LocalOps AI turns your documents and knowledge into a private team of AI assistants.

Users can create agents, give them names and purposes, and assign the sources they should use. Each agent appears as an animated 3D character with its own workspace, task history, knowledge sources, and performance information.

An HR agent can answer questions from an uploaded handbook. A finance agent can inspect a supported invoice ledger for overdue balances and duplicate entries. A support agent can draft responses grounded in assigned documentation. Custom agents can help users explore their own records.

The knowledge library supports document uploads, local OCR, and read-only website or API connections. Sources can be shared across the team or assigned to selected agents.

When users submit a task, they can follow its actual queued, running, and completed states, then inspect the response, cited passages, and recorded timings. A separate local assistant supports questions and follow-up conversations using explicitly selected sources.

The application also includes security controls and an optimizer that reviews recorded workloads and proposes scheduling changes for user approval.

How we built it

We built the frontend with React, TypeScript, Vite, Three.js, and React Three Fiber. The living workspace uses procedural 3D characters, while individual agent pages provide a practical place to manage tasks and review evidence.

The backend uses FastAPI and Pydantic to validate requests and enforce permissions. SQLite stores agent identities, knowledge metadata, jobs, settings, and activity history. Server-Sent Events connect backend execution states to the interface.

For reasoning, we used NVIDIA Nemotron through Ollama, running locally on the ASUS GX10’s NVIDIA GB10 hardware. The inference path uses the local model rather than a hosted model fallback.

We built a knowledge pipeline that extracts text from supported files, indexes it with SQLite FTS5, and retrieves relevant passages for model responses. For supported invoice CSVs, we use deterministic validation and exact decimal arithmetic rather than relying on the model to calculate financial totals.

A durable queue processes one inference request at a time. Multiple agent identities share the same local model, while keeping their knowledge assignments and permissions separate.

Challenges we ran into

One challenge was connecting the visual workspace to real execution. An animated character is easy to show; making its status reflect an actual queued or completed task requires careful coordination between the browser, backend, and persistent storage.

Local inference introduced another challenge: model loading and response times vary. We added a prewarming workflow, explicit task states, and recorded timings so users could understand what was happening.

Document ingestion also required more care than simply accepting uploads. Scanned PDFs need OCR, some files are only partially readable, and remote sources can become unavailable. We made reading status, extraction limits, and cached-source timestamps visible.

Finally, we had to separate helpful model output from trusted system behavior. Retrieved documents cannot grant tool permissions. Authorization happens in backend code, and financial calculations use validated data.

Accomplishments that we're proud of

We are proud that LocalOps AI combines an expressive interface with working local inference and inspectable results.

The repository records real Nemotron requests on the GX10, including an agent answering questions from uploaded project documentation and a local assistant handling a contextual follow-up.

We also implemented reusable knowledge sources, individual agent workspaces, persistent task history, source access controls, and deletion with undo and recovery.

The latest recorded verification includes 107 passing backend tests, along with successful TypeScript checks and a production frontend build.

What we learned

We learned that an AI workforce needs more than multiple personalities. Each agent needs a clear purpose, relevant knowledge, explicit permissions, and visible execution history.

We also learned that citations help users inspect an answer, but do not guarantee that every claim is correct. One recorded response misinterpreted dependency version ranges; a follow-up corrected it. That reinforced the importance of preserving evidence and supporting review.

Building on local hardware taught us to distinguish shared device measurements from individual request metrics. Multiple agents sharing one model do not each have a separate GPU allocation.

What's next for LocalOps AI

Next, we want to improve retrieval quality, evaluate answer accuracy more systematically, and complete a full disconnected-network acceptance test.

We also want to add authenticated provider integrations, stronger authentication, multiuser isolation, and more capable workflows with carefully enforced permissions.

Our long-term vision is to make a private AI workforce practical: specialized teammates powered by your knowledge, running on hardware you control, with work you can inspect.