FieldMind

Your private, on-device AI assistant that keeps working even when the internet doesn't.

Comment

Inspiration

AI assistants have become incredibly capable, but most still depend on a constant connection to powerful cloud servers. Lose connectivity and much of that intelligence disappears. Using cloud AI can also mean sending conversations, documents, and personal information away from the user's device.

We started FieldMind with a simple question:

What if useful AI could live with you instead of always living in the cloud?

That became FieldMind, an offline-first AI assistant designed to run AI models directly inside the user's browser and continue being useful even when the internet isn't available.

Our goal is not simply to build another chatbot. We want to explore what a personal AI assistant looks like when local processing is the default, connectivity is an enhancement, and the user's device becomes the primary computing environment.

FieldMind — Intelligence where the internet isn't.

What it does

FieldMind is an installable offline-first AI assistant capable of running language models directly on the user's device.

Users can download a compatible local model once and then generate responses on-device using WebGPU without sending prompts to a cloud LLM API.

FieldMind currently supports a library ranging from lightweight SmolLM2 360M to Qwen 3 models at 0.6B, 1.7B, 4B, and 8B, allowing different devices to use different levels of local intelligence.

FieldMind can also:

  • 💬 Run conversations using on-device language models
  • 🧠 Remember relevant information across conversations
  • 📄 Read and extract information from PDFs, DOCX, XLSX, text, CSV, and source-code files
  • 📷 Accept images and camera input
  • 🔎 Perform local OCR
  • 👁️ Generate local image descriptions and detect common objects
  • ✂️ Crop, resize, rotate, and prepare photos locally
  • 🎙️ Accept voice dictation where supported by the browser
  • 💾 Preserve conversations locally
  • 🔄 Synchronize conversation history when connectivity returns
  • 📱 Run as an installable Progressive Web App
  • ⚡ Recommend models based on available device capabilities

The result is an AI experience where the model itself can continue generating answers even when the network disappears.

How we built it

FieldMind is built with Next.js, React, TypeScript, Material UI, SQLite/libSQL, Drizzle ORM, IndexedDB, Serwist, WebLLM, WebGPU, Transformers.js, and Tesseract.js.

We designed it around a local-first architecture.

                    FieldMind
                        │
              ┌─────────┴─────────┐
              │                   │
         Local Device          Server
              │                   │
       ┌──────┼──────┐            │
       │      │      │            │
   Local AI  Chats  Files      Account &
       │      │      │         Chat Sync
       │      │      │            │
    WebGPU IndexedDB IndexedDB  SQLite/libSQL
       │      │      │            │
       └──────┴──────┘            │
              │                   │
          Works Offline ──────────┘
                  Sync when online

Local AI

Language-model inference runs directly inside the browser using WebLLM and WebGPU.

The model executes inside a Web Worker so inference does not have to run directly on the main UI thread.

Users can choose between multiple models depending on their hardware. FieldMind inspects available WebGPU support, GPU buffer limits, browser-reported memory, CPU cores, and available browser storage to help recommend an appropriate model.

Model weights, tokenizers, configuration, and required runtime data are cached locally after download. FieldMind also requests persistent browser storage to reduce the likelihood of downloaded AI assets being automatically removed.

Once a model has been downloaded successfully, it can be loaded again locally — including after returning to FieldMind offline.

No cloud LLM API or API key is required for normal model inference.

Local-first conversations

Conversations are written immediately to IndexedDB rather than waiting for a server response.

FieldMind maintains a transactional synchronization outbox. If a message is created while offline, the local copy remains usable and the pending database operation can be retried when connectivity returns.

When online, conversation text and memory information can synchronize with FieldMind's SQLite/libSQL backend.

This means losing connectivity does not mean losing the conversation the user is currently working on.

Offline files

Original attachment binaries are retained locally on the device rather than being uploaded as part of conversation synchronization.

FieldMind can locally extract information from supported documents including PDFs, DOCX, XLSX, text files, CSV files, and source code.

Extracted information can then be supplied to the local language model as context.

Local vision and OCR

Images can be processed using multiple local capabilities.

Tesseract.js provides OCR, including configurations for English, Hindi + English, Bengali + English, Spanish, and French.

For visual understanding, FieldMind uses Transformers.js with quantized browser-compatible vision models for image captioning and common-object detection.

The resulting visual description or extracted text can then be passed to the selected local chat model.

Memory

Language models do not automatically remember a user after a conversation ends, so we built memory as part of FieldMind rather than relying on the model itself.

FieldMind constructs context using:

  • Saved memory notes
  • Recent conversation turns
  • Relevant excerpts from earlier conversations
  • Explicit information the user asked FieldMind to remember

Relevant older information is retrieved across the user's conversations and supplied to the local model when useful.

The current retrieval system is lexical and intentionally separate from the model, meaning users can change local models without losing their conversation history or saved context.

Challenges we ran into

One of our biggest challenges was discovering that running AI offline is not simply a matter of downloading a model.

Different devices expose dramatically different GPU capabilities, memory limits, storage quotas, and WebGPU features.

A model that works comfortably on a powerful desktop may be unusable on a phone.

Instead of choosing one model for everyone, we built a device-aware model library ranging from 360M parameters to 8B parameters and evaluate the device before recommending an option.

Another challenge was browser storage.

AI models can require hundreds of megabytes or several gigabytes of local data. We had to combine IndexedDB caching, persistent-storage requests, service-worker caching, and careful model lifecycle management to make returning to an already-downloaded model practical.

Offline synchronization introduced another problem: users can continue modifying conversations while the server is unreachable.

We solved this using local transactions and an outbox that records pending writes and retries them after reconnection. We also added deletion tombstones so an older offline device cannot accidentally resurrect a conversation that has already been deleted.

Memory was another challenge. The local model itself has no permanent memory, so we had to create our own retrieval system around durable conversations while remaining within the model's finite context window.

Finally, multimodal input required us to combine several very different technologies — document parsers, OCR, vision models, WebGPU language models, local file storage, and browser APIs — without turning every interaction into a cloud request.

Accomplishments that we're proud of

We're particularly proud that FieldMind's offline-first concept is not only a UI promise.

The language model actually runs on the user's device.

A user can download a model, disconnect from the internet, return to the application, load the cached model, access locally stored conversations, and continue generating responses on supported hardware.

We're also proud of building:

  • A device-aware local model library from 360M to 8B parameters
  • On-device Qwen 3 inference using WebGPU
  • Persistent local conversation storage
  • Offline conversation creation and recovery
  • Automatic synchronization after reconnection
  • Cross-conversation memory retrieval
  • Local document extraction
  • Local OCR
  • Local image captioning and object detection
  • Device-local original attachment storage
  • Local photo preparation tools
  • An installable PWA experience
  • Guest-to-account conversation migration
  • Conversation deletion reconciliation across offline devices
  • Long-response continuation beyond individual inference context boundaries

Together, these features helped us move FieldMind from the idea of an "offline chatbot" toward a real local AI workspace.

What we learned

The biggest thing we learned is that the language model is only one component of a useful AI assistant.

A practical local assistant looks much more like:

Local AI
+ Local Storage
+ Memory
+ Documents
+ OCR
+ Vision
+ Synchronization
+ Device Detection
+ Offline Application Infrastructure

We also learned that bigger models are not automatically better products.

A smaller model that actually runs reliably on someone's device can be much more useful than an enormous model that cannot load at all.

This led us to treat models as replaceable resources rather than hard-coding FieldMind around a single LLM.

We also learned how difficult offline synchronization can become. Once both the device and server can hold state, reconnection, ownership, retries, deletion, partial responses, and interrupted sessions all have to be handled carefully.

Most importantly, FieldMind taught us to think about where computation happens, not simply which AI API to call.

What's next for FieldMind

FieldMind already demonstrates local language-model inference, memory, offline conversation storage, document processing, OCR, vision capabilities, and synchronization.

Next, we want to make those capabilities even more powerful.

Our roadmap includes:

  • 🔎 Semantic/vector-based memory retrieval
  • 📚 More advanced local document RAG
  • 🎙️ Fully offline speech recognition and text-to-speech
  • 👁️ More capable multimodal local models
  • 📦 Downloadable Field Packs for education, agriculture, repair, travel, and other specialized domains
  • 🔧 Safe deterministic local tools and actions
  • 🤖 Offline agent orchestration for complex multi-step tasks
  • 🔐 Stronger client-side privacy and encrypted synchronization
  • 📱 A native mobile implementation sharing FieldMind's core architecture
  • ⚡ Continued optimization for lower-powered hardware

Our long-term goal remains simple:

FieldMind should be able to see, hear, read, remember, reason, and help while keeping as much intelligence as possible directly on the user's device.

AI should not stop being useful because connectivity disappears.

Intelligence where the internet isn't.

Built With

  • computervision
  • drizzleorm
  • huggingface
  • indexeddb
  • localai
  • materialui
  • next.js
  • ocr
  • offlinefirst
  • ondeviceai
  • pwa
  • qwen3
  • react
  • restapi
  • serwist
  • smollm2
  • sqlite
  • tesseract.js
  • transformers.js
  • typescript
  • webassembly
  • webgpu
  • webllm
Share this project:

Updates

Submission history