8 Milliwatts

Piano demonstration
Projection Helmet

Instead of augmenting reality through traditional AR glasses, 8 Milliwatts projects guidance directly onto the physical environment. The system uses a helmet-mounted projector and lasers to overlay instructions onto real-world surfaces, enabling people to learn skills in a more physical, intuitive way.

8 Milliwatts is a wearable AR projection-based guidance system designed for accelerated skill acquisition—from learning piano to assembling machinery and operating industrial equipment. Rather than relying on isolated personal displays, the system projects guidance directly onto physical surfaces, making the learning process visible, intuitive, and collaborative.

The platform integrates real-time spatial mapping, voice interaction, and visual feedback into a unified perception-action pipeline. Both the helmet-mounted camera and the Mentra Glass camera detect AprilTag markers to localize and map target surfaces, enabling robust tracking of the workspace. Using these visual inputs, the system computes projector coordinate transforms that allow pixel-accurate overlays aligned with the physical environment.

A session state machine manages the lesson flow, while Mentra Glass handles voice command recognition and provides a live camera feed of the workspace. Spoken commands such as next, repeat, and stop are interpreted and dispatched to a central orchestrator, enabling hands-free control during instruction. The camera feed is transmitted to a laptop for real-time analysis, where computer vision detects AprilTag markers and tracks user actions. Modle (k2thinkv2) processes the visual inputs to infer the current task state and update the instructional flow. Laser projection then renders guidance symbols directly onto the workspace, providing immediate visual feedback for each step of the task.

While the demonstration focused on piano instruction, 8 Milliwatts’ architecture is fully domain-agnostic. The same spatial mapping and projection pipeline can guide technicians through equipment assembly, train factory workers on machinery, or support any hands-on skill where instructions can be projected directly into the physical workspace.

Collaborators:
Mohammed Aamir
Keira Boone Nick Nim
Chris Um
Cecilia Xu
Rachel Yang
Anna Zhang
Gloria Zhu