Inspiration Most AI assistants today depend on large cloud models. They are powerful, but they require an internet connection, send personal context to external servers, and are difficult to customize around a single user. I wanted to explore a different question: Can we build a useful personal language model from scratch, make it small enough to run on a phone, and still allow it to take real actions across everyday applications? That idea became SelfLM — a scratch-trained, roughly 85-million-parameter language model designed to run locally on a laptop or Android phone, understand the user's requests, and interact with tools such as calls, messages, and contacts. SelfLM is not intended to compete with frontier models on general knowledge. Its purpose is to become a focused, private, low-latency personal intelligence layer that belongs to the user. What SelfLM Does SelfLM combines an on-device language model with a secure tool-proposal system. The model can interpret requests such as:
- "Show me the important emails I received today."
- "Draft a reply to this recruiter."
- "Check my calendar and find a free slot tomorrow."
- "Send this message through WhatsApp."
- "Find this person in my contacts." Instead of directly accessing applications, the model produces a structured tool proposal. For example: { "tool": "gmail.search", "arguments": { "query": "is:unread newer_than:1d" } } A separate deterministic host layer classifies the request, validates it, and prepares a call or message proposal — but it never executes that proposal directly. The phone-side layer must resolve contacts and request native confirmation before anything is sent, dialed, or opened. This separation keeps the model small while making the overall system capable, and it keeps identity, credentials, and execution outside the model so an 85M model is never trusted with safety-critical decisions. How I Built It SelfLM is a decoder-only Transformer trained from scratch rather than a wrapper around an existing hosted language-model API. The tokenizer and neural weights both begin from random initialization.
- A custom model architecture The backbone is a FrontierSelfLM decoder-only causal language model with modern, evidence-selected components:
- Grouped-query attention with RoPE positional encoding and a sigmoid gated output projection, selected over a KDA/GQA hybrid and a MobileLLM-Flash-inspired variant after a matched 200M-token-per-arm experiment. Standard GQA delivered similar held-out language quality with substantially higher throughput, so it became the qualified control.
- RMSNorm (not LayerNorm) for pre-norm residuals.
- SwiGLU gated feed-forward network (gate + up + down projections with SiLU activation).
- Optional per-head Q/K RMSNorm (Qwen3-style) for the mobile-flash candidate.
- Shared-NEXTN MTP-2 (multi-token prediction) draft module, the DeepSeek-V3-style shared embedding draft head that predicts two tokens ahead. At the 600M-token gate it preserved base-model quality and produced a 1.969x measured end-to-end speculative-decoding speedup (95% lower bound 1.794x, 4.34% peak-memory overhead). Every accepted speculative token was verified by the target model — this was not an unverified draft-only timing. The submitted 85M-tier checkpoint is 95.6M parameters including the MTP-2 heads: 16 layers, 640 hidden width, 1536 FFN width, a 16,384-token custom BPE vocabulary, and a 2,048-token context window. The codebase also defines 139M and 355M mobile tiers for later deployment.
- A custom tokenizer A byte-level BPE tokenizer with a 16,384-token vocabulary was trained from scratch. It includes 13 special control tokens for the chat and tool format (<|system|>, <|user|>, <|assistant|>, <|tool|>, <|observation|>, <|act|>, <|direct|>, <|deliberate|>, and document/contact separators), so conversation turns and tool roles are first-class to the model rather than injected as plain text.
- Foundation pretraining The model completed about 4.29B accepted pretraining tokens across four exact-resume segments. Held-out NLL improved from 2.891 to 2.624 and perplexity from 18.013 to 13.795 while throughput stayed near 298K accepted target tokens per second. Every expensive stage was gated by canaries, exact resume, sealed evaluation, retention, and rollback checks — no stage was promoted by training loss alone. The optimizer was selected by a 100M-token proxy: Muon with auxiliary AdamW improved held-out NLL 3% relative to the AdamW control while staying within the registered 10% throughput budget.
- Compact chat recovery training The first chat SFT lowered its own development loss while seriously damaging foundation retention and product-chat behavior. Those checkpoints are retained as failure evidence and are not release candidates. The completed recovery path built a 450K-example compact chat pool from pinned, licensed public datasets plus 460 independently reviewed seed examples:
- 145K from SmolTalk
- 185K from Nemotron instruction-following chat v1/v2/v3
- 70K from UltraChat 200K
- 49,540 from Tulu-3 FLAN v2 Public answers were filtered and selected; they were never silently rewritten to satisfy length quotas. Length was stratified (80% short ≤64 target tokens, 15% medium, 5% long) with a 10% long-context reservation. The final run used 405,007 training rows, kept MTP frozen, and ended at optimizer step 4,745.
- Structured tool calling and the host runtime SelfLM is trained to decide whether a request can be answered locally or requires an external tool. When a tool is needed, the host runtime produces a strict, validated proposal rather than letting the model emit unrestricted text. The runtime is a loopback-only HTTP server (bound to 127.0.0.1) with a built-in chat UI. Identity ("who are you?") and tool execution are deterministic host responsibilities — the 85M model is never trusted to remember who it is or to safely execute a phone action. Calls and messages are proposals until a native layer validates them and asks for confirmation.
- Mobile-focused inference The codebase defines 85M, 139M, and 355M mobile tiers and includes ExecuTorch export qualification and cached-decode paths for on-device deployment. The goal is to run SelfLM directly on an Android phone through quantization, reduced memory usage, efficient cached token generation, and the compact 2,048-token context window — without requiring a cloud GPU during normal use. Challenges The biggest challenge was balancing intelligence with size. An 85M-parameter model has limited reasoning capacity compared with cloud-scale models. This means the data, prompts, output format, and tool definitions must be extremely clear. The model cannot rely on brute-force scale; it must learn a smaller and more focused set of behaviours reliably — and even then, the current checkpoint scores 4/10 on the product-chat gate, which is reported openly rather than hidden. Catastrophic forgetting was the central training challenge. The first chat fine-tune destroyed foundation behavior while looking successful on its own loss. Recovery required per-sequence-normalized loss, 25% foundation replay, reference-KL regularization, and MTP freezing — and every candidate was judged on held-out foundation + product-chat gates, never on training loss alone. Late-run GPU failures forced a switch to microbatch accumulation and exact-resume runners, because a naive restart would have discarded hundreds of millions of accepted tokens. The exact-resume system verifies optimizer step, data position, and RNG state before any continuation. MTP-2 added another layer of risk: speculative decoding is only safe if every draft token is verified by the target model. A naive draft-only timing would have overclaimed speedup, so the benchmark counts only verified accepted tokens and reports a paired 95% lower bound. Mobile deployment created additional constraints around RAM usage, model size, token-generation latency, battery consumption, thermal throttling, and context-window length — all of which push toward the smaller tiers even when larger tiers score better offline. Connecting personal applications also introduced security challenges. Credentials cannot be placed inside the model or exposed in prompts, so authentication and execution remain isolated inside the deterministic host layer. The model only ever sees tool proposals and observations, never credentials. What I Learned The most important lesson was that the language model does not need to contain every capability itself. The model's job is to understand intent, select the correct tool, and explain results. Normal software should handle deterministic operations such as sending an email, reading a calendar, looking up a contact, and remembering who the assistant is. Trusting an 85M model with identity or direct tool execution would be both unsafe and unreliable. I also learned that building a language model is not only about architecture. Data quality, tokenization, evaluation, inference efficiency, tool reliability, and safety controls are equally important — and the interesting failures live at the boundaries between them. The first chat run that "won" on training loss but broke the product is the clearest example: a metric can look healthy while the system regresses. A smaller model can become surprisingly useful when it is given a clearly defined purpose, high-quality task-specific training data, reliable external tools, strict permission boundaries, and a well-designed fallback system. MTP-2 speculative decoding showed that even a tiny model can approach larger-model latency when the draft head is cheap and every token is verified. Honest limitations
- Final optimizer step: 4,745.
- Held-out BPB: 0.994 (passes the registered 1.00 ceiling).
- CORE16: 0.1036 (below the 0.105 target).
- Product-chat score: 4/10.
- The checkpoint is a hackathon research artifact, not a release-ready model.
- Identity and tool execution are deterministic host responsibilities by design. What's Next The next stage is to improve instruction-following and tool-selection accuracy, expand mobile quantization support, and add more locally processed personal context. My long-term vision is for SelfLM to become a private AI operating layer for the phone — one that can work offline whenever possible, securely use connected applications when necessary, and adapt to its owner without continuously sending their life to the cloud. SelfLM is an attempt to build AI in the opposite direction of today's largest systems: smaller, personal, local, and owned by the user.
Log in or sign up for Devpost to join the conversation.