Inspiration
In a world moving rapidly toward expensive, privacy-compromising cloud AI APIs, we wanted to build something fundamentally different: a fully private, offline "second brain" that actually respects user privacy.Most AI productivity tools require powerful GPUs or heavy cloud subscriptions. We wanted to build NeuroLayer AI for the everyday developer, student, or researcher running standard consumer hardware—proving that you can have a highly intelligent, semantic knowledge base right on an older 8GB RAM laptop without sending a single byte of data to the cloud.
What it does
NeuroLayer AI acts as an external cognitive layer running entirely on your machine. It allows you to:
Capture Memories: Seamlessly log notes, links, and text with custom tags, importance ratings, and source tracking.
Semantic Vector Search: Automatically embeds your data in the background using nomic-embed-text and indexes it via FAISS for lightning-fast, context-aware retrieval.
Smart Memory Linking: Automatically detects similarities between distinct notes above a configured threshold to show you how your thoughts connect over time.
Grounded AI Chat: Features a fully offline, streaming chat interface powered by Qwen2.5:3B via Ollama. It utilizes strict RAG (Retrieval-Augmented Generation) to answer prompts only from your retrieved personal context.
Cognitive Dashboard & System Monitor: Visualizes your learning streaks and top tags, while tracking real-time CPU, RAM, and FAISS vector index health.
How we built it
We architected a production-ready, local-first stack optimized entirely for efficiency:
Backend: Built with FastAPI utilizing a strict clean architecture (Repositories, Services, Schemas, and API Routers).
Frontend: A clean, responsive dashboard designed in React (Vite + TypeScript).
Database & Vector Cache: MySQL 8 stores traditional relational data and caches vector embeddings so they never have to be re-computed unnecessarily.
AI & Embeddings Engine: Ollama runs local instances of Qwen2.5:3B and nomic-embed-text.
Vector Index: FAISS (IndexFlatIP) handles exact cosine similarity search without the massive memory overhead of approximate nearest neighbor (ANN) graphs.
Infrastructure: Orchestrated completely via Docker Compose, with secure headers, custom rate-limiting middleware, structured logging, and automated migrations using Alembic.
Challenges we ran into
Building a smooth AI experience inside an 8GB RAM, CPU-only ceiling was a massive exercise in optimization.
Resource Competition: Running MySQL, a Python backend, a Vite frontend, an embedding model, and an LLM simultaneously can easily choke a machine. We resolved this by pinning a single Uvicorn worker (since the LLM bottleneck makes multiple workers a waste of RAM) and picking the heavily quantized Qwen2.5:3B model.
UI Responsiveness: Generation of embeddings can cause noticeable interface lag if done on-request. We built an asynchronous background processing system—writes return instantly to the user while vectors process silently in the background.
Docker Boot Order: During early runs, the backend would frequently try to connect before MySQL finished initializing. We resolved this by tuning Docker health checks and optimizing connection pooling with pool_pre_ping.
Accomplishments that we're proud of
Zero-Cloud Architecture: We achieved a fully localized, end-to-end RAG workflow that runs comfortably alongside normal operating system tasks on a standard 8th/10th Gen Intel i5 processor.
Bulletproof Architecture: The backend features a pristine structural layout with dependency injection and modular services, making it incredibly stable and clean.
Embedding Efficiency: By caching vectors directly in MySQL, we ensure that restarting the application or rebuilding the FAISS index costs almost zero compute overhead.
What we learned
We learned how to tightly manage resources for local AI deployments. We discovered that for personal-scale datasets, sophisticated ANN vector graphs are overkill; using FAISS's exact IndexFlatIP dramatically reduces memory usage while providing perfect accuracy. We also gained deep insights into structuring asynchronous background workers to keep user experiences feeling instantaneous even on heavily constrained hardware.
What's next for NeuroLayer AI
The foundation is perfectly wired, and we have already modeled the database schema for our next cognitive feature tier. Up next, we plan to implement:
Automated AI Refinement: Leveraging the local LLM for automated note-tagging and importance scoring.
Cognitive Optimization: Activating the spaced-repetition review schedules and daily/weekly reflection summary modules.
Smart Ingestion & Portability: Building chunking tools for PDF, Markdown, and JSON imports, along with clean Markdown export routines.
Built With
- alembic
- docker
- docker-compose
- faiss
- fastapi
- jwt
- mysql
- ollama
- python
- qwen2.5
- react
- sqlalchemy
- typescript
- vite
Log in or sign up for Devpost to join the conversation.