Inspiration
A Field Service Engineer with their hands deep inside a server rack will not have time to use a laptop or be distracted from valuable work if they did so. In the enterprise B2B space, augmented reality is often treated as a gimmick, just floating 2D PDFs in the air. I wanted to build a true AR app that is probably better than its laptop version
What it does
The inspiration was to create a closed-loop spatial operating system. I already had two existing backends: BlocTi (an event-sourced labour and execution tracker) and stoQr (an automated inventory and procurement API). I realised that if I combined them inside Snap Spectacles (SPECS), the physical act of repairing a machine could simultaneously become the data entry. No keyboards, no barcode scanners, and no typing
How we built it
We used Lens Studio paired with the ls-clad (Closed Loop Agentic Development) plugin to rapidly prototype the spatial environment and interaction mechanics.
The core experience centres around swapping a failed Power Supply Unit (PSU) in a 1:1 virtual blade server rack. Instead of passive buttons, I focused on spatial logic:
The Physical Gate: The engineer must grab a virtual PSU and physically drag it into Slot 2. The confirmation button remains disabled until the 3D model snaps into place (within a 5cm tolerance).
The Dual-Backend Architecture: Once the swap is confirmed, the Lens triggers our DualBackendManager.js. To make the invisible APIs visible for the pitch video, we built a "System Audit Log" UI that slides out to display the mock payloads:
stoQr instantly deducts the part and queues a reorder.
BlocTi calculates execution fidelity based on our event-sourced projection math. For example, our backend calculates time drift using LaTeX-represented logic
$$\text{Fidelity} (\%) = \left( 1 - \frac{\vert{}T_{\text{actual}} - T_{\text{planned}}\vert{}}{T_{\text{planned}}} \right) \times 100$$
We built the network scripts behind a USE_MOCK_API = true toggle, simulating latency and offline-sync fallbacks.
Challenges we ran into
The Additive Display Trap: Initially, we envisioned sleek, dark semi-transparent UI panels. We quickly realised that on the SPECS additive displays, dark colours render as transparent and completely disappear. We had to pivot our entire design language to glowing white wireframes with high-contrast cyan text to ensure readability against physical backgrounds.
FOV Ergonomics: The Spectacles have a specific field of view (~37° tall). We initially placed our checklist 30cm to the right of the rack. We quickly learned this forced the engineer to turn their head back and forth, breaking immersion. We had to cluster the interactive zones (the PSU slot, the power button, and the UI) into a tight 50 × 75 cm window at chest height, angled 15° inward to sit perfectly in peripheral vision.
Automated Testing Conflicts: When generating our automated interaction tests, the ls-clad builder initially tried to install LEAF 4.0.0 (designed for Lens Studio 5.25). This caused massive compile errors on our 5.23 build. We had to manually intervene by downgrading the package to LEAF 2.0.2 and rewriting the DefaultLeafInteractor scenarios. Ultimately, we achieved passing tests across all 5 of our core done criteria.
Accomplishments that we're proud of
100% Automated Test Pass: We achieved a 5-out-of-5 pass rate on our LEAF suite driving real SIK? interactions.
Closed-Loop Dual-Backend Integration: We successfully merged two distinct platforms—BlocTi for labour tracking and stoQr for procurement—into one spatial system.
Enforced Operational Fidelity: The physical snap gate keeps confirmation buttons disabled until the virtual part snaps into place, preventing falsified repair logs.
Hardware-First Spatial Design: We engineered light-on-dark wireframes and compact, chest-height layouts built specifically for additive displays and real-world field-of-view constraints.
Bulletproof Architecture: We built a reliable offline fallback system that protects backend intellectual property while guaranteeing a zero-failure demo video.
What we learned
I learned that spatial computing isn't about giving the user more screens; it is about giving them fewer abstractions.
Building this taught me that the most powerful AR interactions are gated. When you restrict a UI button until a physical part snaps into a physical slot, you enforce 100% operational fidelity. I also learned how to "design for the preview" using stand-ins like the power-button pinch to simulate reading machine state, allowing me to prove out complex B2B logic without needing a physical server rack in my living room
What's next for Guided Maintenance
Physical Computer Vision Integration: Transition from Surface Tracking stand-ins to Snap’s Object Tracking and Custom Landmarker APIs to recognise physical server chassis models automatically without manual anchor placement.
Live Telemetry & IPMI Connectivity: Replace simulated diagnostic steps with direct SNMP/IPMI network polling from real server racks, automatically triggering AR overlays when hardware faults occur.
Production API Deployment: Toggle USE_MOCK_API = false and deploy our Supabase Edge Function proxy to securely connect to live stoQr procurement tables and BlocTi event logs without exposing client API credentials inside front-end Lens packages.
Multi-Technician Remote Collaboration: Integrate real-time spatial anchor sharing so remote tier-3 engineers can draw annotations into the field service engineer’s Spectacles viewport during complex repairs.
Built With
- clad
- claude-code
- javascript
- lens-studio
- typescript
