YazSes Spatial Review

A SPECS Lens that shows a simulated transcription as an unconfirmed draft beside a tabletop task, so you explicitly confirm or cancel it. Simulated input only, tested in Lens Studio Preview.

Comment

Inspiration

Dictation tools put text somewhere you are not looking: a window on another screen or a field in another app. When your hands and eyes are busy with a physical task, checking what was heard means looking away, and a wrong sentence can be accepted by accident. We wanted to explore one small question: what would a review step look like if the draft sat in the scene next to the task, and nothing counted until you deliberately confirmed it?

This is a research prototype of the review step only. The input is simulated. The Lens always shows "Simulated transcription — not live speech". It does not capture speech, run speech recognition, connect to the YazSes dictation software, or send anything anywhere, and it has not been tested on physical glasses. YazSes is the author's separate offline dictation project; this Lens is not integrated with it.

What it does

A virtual tabletop workbench (about 60 x 40 cm, 1.1 m ahead) holds a blue and a yellow 8 cm block. A note card floats at a fixed position beside the workbench with its controls directly beneath it:

  1. Place note shows the note beside the blocks.
  2. Choose Simulated transcription A ("Prepare the blue and yellow blocks.") or B ("Place the blue block beside the yellow block."). Both are fixed preset strings. The selection becomes an unconfirmed draft ("Draft — not confirmed"); switching between A and B never confirms.
  3. Confirm (visually separated from Cancel) changes the status to "Confirmed". A repeated Confirm has no extra effect. Cancel discards the draft and confirms nothing.
  4. Reset returns to a clean placed note.

Controls that are not valid in the current state are hidden; the selected sample button stays visible but inactive. The note keeps its position beside the workbench as the viewpoint moves and turns about its vertical axis to face the viewer (a yaw-only Spectacles Interaction Kit Billboard); it is not head-locked.

How we built it

  • Planning: the design brief is in the repo (lens/YazSesSpatialReview/devpost/design-brief.md), with five Preview-checkable done criteria.
  • Building: Snap CLAD and the Lens Studio MCP, driven from Claude Code with Lens Studio 5.24.1 open, created and wired the scene, compiled and ran the TypeScript, configured and captured the SPECS 27 Preview, and ran the tests.
  • Logic: the review logic is a small pure TypeScript state machine (IDLE, PLACED, DRAFT, CONFIRMED, CANCELLED; events place, select sample, confirm, cancel, reset, interrupt) with no Lens, browser or Node dependency. It has Vitest unit tests, and the same source files compile unchanged inside the Lens project. The UI script only renders the logic's view model.
  • Tests: five LEAF scenarios, one per done criterion, drive the real UI Kit buttons in the Lens Studio Preview. All five passed on their final runs (details, including the failed first run of the viewpoint test, are in docs/TESTING.md).
  • Parts of the code were written with an AI coding assistant (Claude Code); the human author directed and reviewed the work.

Challenges

  • Making the confirm step safe: no transition may confirm without an explicit Confirm while a draft is open, including after a switch, reset or interruption.
  • Package versions: newer Asset Library packages (UI Kit, LEAF 4.0.0, the preview agent) import a newer SIK package name than the Base Template's, which broke compilation. We kept the template's packages, used LEAF 2.0.2, and removed the incompatible add-ons.
  • Injected Preview taps do not reach the SIK mouse interactor, so button presses are tested through LEAF's interactor.

Honest limitations

  • Simulated input only: two preset strings. No microphone, no speech recognition, no live transcription, no network.
  • Not integrated with YazSes; no YazSes code is used.
  • Verified only in Lens Studio Preview (SPECS 27, Interactive). Comfort, readability and hand tracking on real glasses are unproven. No device test was run.
  • The viewpoint test moves the camera by script with tracking suspended; rotation toward the viewer was checked by eye, not asserted.
  • No user study and no evidence of any accessibility benefit.

What's next

Study whether a spatial review step reduces accidental acceptance compared with a flat window; test on real hardware if it proves useful; only then consider a real transcription source, as a separate design with its own privacy and consent review.

Built With

  • clad
  • claude
  • code
  • interaction
  • kit
  • leaf
  • lens
  • mcp
  • snap
  • specs
  • spectacles
  • studio
  • typescript
Share this project:

Updates

Submission history