OA Mobile Lab — Local AI Training Across Vulkan GPUs

Inspiration The accelerator you already own should be able to train. A useful local-AI application rarely ends with model inference. It captures or decodes data, transforms it, trains or runs a model, evaluates the result, saves its state, and then displays, plays, encodes, or streams the output. Those stages are usually assembled from several frameworks. Each boundary can introduce another data model, allocator, device policy, copy, synchronization mechanism, build system, and vendor-specific backend. The developer ends up maintaining the seams as well as the product. OA began with a question: can matrices, images, audio, video, and model state retain their own semantics while sharing one explicit GPU execution foundation? Integrated and mobile GPUs are central to that idea. They are programmable accelerators already inside devices people own, yet they are often treated as display hardware or inference-only targets. We wanted to demonstrate complete, inspectable training workflows on them. What It Does OA is a GPU-first C++ framework with Python bindings, built around a capability-driven Vulkan runtime. It provides foundations for numerical computing, machine learning, Vision, Audio, media, reinforcement learning, plotting, and compact rendering without exposing a separate high-level backend for every GPU vendor. One OaEngine owns device selection, memory, queues, kernels, scheduling, and profiling. Stateless operations describe semantic transformations, while training, decoding, encoding, streaming, and presentation remain explicit stateful sessions. Semantic work is lowered into concrete dispatch, transfer, synchronization, aliasing, and completion records. Typed values can share device storage when their representations and lifetimes permit it while preserving the distinction between a matrix, image, audio buffer, video frame, or mesh. The Build Week proof is OA Mobile Lab. On the reference Qualcomm Adreno 610 phone, the signed Android application locally trains five small neural-network architectures: RNN GRU Transformer Sparse-MoE Transformer Mamba-3 A route does not pass simply because its loss decreases. It must complete Vulkan forward execution, loss calculation, reverse-mode autograd, AdamW optimization, evaluation, autoregressive generation, checkpoint saving, fresh-model recreation, checkpoint loading, and exact post-reload generation parity. The equivalent controlled training contract also runs on an Intel Iris Xe development system. This demonstrates the architecture on two materially different GPU and driver stacks; it is not a claim that every Vulkan device behaves identically. How We Built It OA combines several layers in one repository: C++20 values, operations, modules, autograd, optimizers, checkpoints, and runtime services Vulkan compute with explicit resource ownership and synchronization Slang shaders compiled into SPIR-V kernels Capability-driven internal kernel selection Recorded and compiled graph execution Declarative operation schemas and generated API surfaces Nanobind Python modules over the native C++ library Android foreground execution, cancellation, live metrics, and signed packaging Linux runtime, SDK, distro-package, and Python-wheel release automation Desktop and Android use the same teaching corpus, dimensions, optimizer semantics, prompt, update count, learning gates, generation checks, and .oam checkpoint format. Mobile is not maintained as a separate inference implementation. OA existed before Build Week, so we kept the baseline explicit. Version 0.7.2 was the pre-event engineering foundation. During Build Week, we: built OA Mobile Lab as a coherent Android product; unified desktop and physical-phone validation; added packed Transformer and GPU-authored sparse-execution paths; introduced bounded backward routes for mobile hardware; reduced controlled sparse-MoE overhead from 46% to 8% for Byte and from 46% to 12% for BPE workloads; added deterministic completed-execution reports; added real APK signing, dependency checks, generated-source drift detection, and sanitized public-release automation. The maintained result is the public v0.7.6 preview, published with matching source, Linux packages, a CPython wheel, signed APK, demo video, and complete checksums. How Codex and GPT-5.6 Helped OA is developed by one engineer working with Codex. GPT-5.6 in Codex acted as an engineering collaborator across repository audits, architecture review, implementation, differential diagnosis, test construction, documentation, and release automation. We followed a repeated evidence loop: Inspect the live call path and ownership model. State the correctness invariant. Implement the smallest complete vertical change. Run a focused oracle. Rerun the relevant desktop, mobile, and cross-module gates. Codex was especially useful for tracing relationships across a large C++, shader, Python, Android, documentation, and build-system codebase. Hardware results still came from executing OA on the referenced devices; they were not generated benchmark claims. Challenges Training is a larger portability contract than inference Inference can freeze parameters and pre-plan a forward graph. Training adds saved activations, gradients, mutable parameters, optimizer state, random state, checkpoint recovery, and many additional synchronization edges. That is why generation and fresh checkpoint recreation became mandatory acceptance gates. A model that reports a low teacher-forced loss but produces corrupt output is not considered successful. Different drivers expose different mistakes One of the most valuable failures appeared only on the physical phone. A packed Transformer projection produced corrupt learning behavior through Turnip even though the desktop path appeared to work. The problem was a bindless descriptor selection that varied between shader threads without satisfying the required uniformity contract. We corrected the shader, rebuilt and installed the signed application, reran the complete phone suite, and then reran the Intel regressions. The lesson was direct: “works on my GPU” is not a portability test. Small models expose orchestration costs Teaching-size networks do not spend all their time inside large matrix multiplications. Dispatch overhead, barriers, host synchronization, and temporary allocation can dominate useful arithmetic. Improving this required graph replay, packed kernels, sparse-route compaction, memory reuse, pipeline caching, and better visibility into fallback behavior—all while preserving numerical, learning, generation, and checkpoint contracts. Breadth made the story difficult to communicate OA covers compute, ML, Vision, Audio, media, RL, and rendering foundations. Trying to demonstrate every module equally would have obscured the project. Mobile Lab became the central proof: one demanding physical result that exercises the same runtime used by the broader framework. What We Learned Cross-vendor correctness cannot be inferred from a single desktop driver. Capability queries are necessary, but they are not substitutes for running complete workloads on real hardware. We also learned that training loss alone is a weak end-to-end signal. Generation, save/reload, fresh-model restoration, deterministic execution reports, and physical-device testing found defects that narrower tensor comparisons missed. The architecture became clearer when ownership and completion were explicit: one engine owner, typed semantic values, stateless operations, explicit sessions, and observable completion. Most importantly, AI-assisted development worked best when confident output was treated as a hypothesis. Source code, specifications, generated-code checks, independent oracles, validation output, and reproducible hardware runs remained the final authority. Accomplishments Five signed-release Android routes passed 300-step training, evaluation, generation, checkpoint save, fresh reload, and exact generation parity. The controlled Transformer and sparse-MoE contracts also passed on Intel Iris Xe. A physical-phone gate found a shader defect that desktop testing had tolerated. Python and C++ use the same native operation library. Completed execution can be inspected through implementation IDs, kernel hashes, resource lifetimes, alias groups, barriers, and completion timelines. Judges can test the project through a signed APK, Python wheel, or packaged Linux runtime without rebuilding the framework. What’s Next OA remains a development preview. Its APIs and model formats may change before 1.0. FP32 is the fully exercised path on the reference hardware, codec support remains device- and profile-dependent, and the educational NLP corpus demonstrates executor correctness and learning behavior rather than production language quality. The next milestone is deeper architectural convergence and stronger persistent hardware validation: stabilizing the public surface, completing schema-driven operation generation, expanding real-GPU CI, and continuing to prove complete vertical slices before claiming broader support. Repository: github.com/realminc/oa Release: OA v0.7.6 Documentation: dev.realm.software

Built With

• c++
• java
• python
• vulkan