Inspiration

Learning to sew from a video is awkward. Both hands are on the fabric, the needle is moving and the moment you need the next step ("Backstitch now!") is exactly when you can't reach for your phone or scroll a page. A beginner ends up looking away from the machine, stopping mid-seam and losing the line. We wanted the lesson to live where your eyes already are: on the machine, right at the needle.

What it does

SeamLens is a hands-free AR sewing tutor for Specs. It walks a beginner through setting up a machine and sewing a first straight seam, with every guide pinned to the machine's needle:

  • Setup: places the machine on your table and gives you the first prep tip (raise the take-up lever with the handwheel).
  • Thread: shows arrows one at a time along the thread path: spool → tension → take-up lever → needle, each with a short tip.
  • Bobbin: marks the drop-in bobbin case, shows a turning arrow for the correct unwind direction, then cues you to pull the thread through the slot.
  • Position: lays a glowing 5/8" (1.5875 cm) seam allowance line on the table at real size, right of the needle.
  • Sew: as the fabric feeds, cues fire automatically: Backstitch now → Keep the edge on the line → Slow down → Backstitch now → Lift the presser foot. No buttons needed.
  • Finish: the line fades out and the lesson wraps up.

You navigate with your voice ("next", "back", "pause", "repeat") or a pinch. If the guides drift, say "recalibrate" (or hold a pinch for 1 second) and everything snaps back to the needle, with a confirmation right where you're looking. Only the current step's guidance is ever visible and text sits on a small card beside the seam line, never in front of your face.

How we built it

  • Lens Studio with TypeScript, targeting Specs, built and demoed in Lens Studio's Interactive Preview.
  • Spectacles Interaction Kit and UI Kit for pinch targets, the hold-to-recalibrate button, and the step card, so it works with a real hand or a mouse.
  • ASR (speech recognition) for voice commands, with pinch as the fallback for every action.
  • A single needle anchor: every guide (arrows, seam line, bobbin markers, cues) is positioned relative to it, so recalibrating moves everything at once.
  • A clearly labelled stand-in sewing machine built in code from simple shapes, with a moving needle bar, spinning handwheel and lifting presser foot, plus a simulated fabric feed that drives the Sew cues.
  • LEAF (Lens Studio's live test framework): 7 automated scenarios run in Preview, covering every done criterion plus the bobbin step and the voice path. All 7 pass.
  • Built with an AI coding agent (Claude Code) from a written design brief, iterating on design notes rather than hand-written code.

Challenges we ran into

  • Sign-in blocked two features in Preview. Without a working login, live speech recognition and the AI 3D model generator were unavailable. We drove voice commands through the same code path in tests, gave every voice command a pinch button and built the machine stand-in from primitives.
  • Shaders stopped building mid-project. After a cancelled project load, Lens Studio deleted 179 cache folders as "unused", including compiled shaders. Its shader compiler then kept timing out ("Cross-compiler process failed to finish"), so the UI Kit's rounded-panel shader came out incomplete. The UI Kit then threw "OpacityFactor uniform is missing from material" on every frame and the whole Lens stopped rendering. We rebuilt the step card, buttons, pause control and confirmation in a "plain mode": text, simple panels and pinch buttons made from the project's own working shader plus the Interaction Kit.
  • Downgrading wasn't an option. We tried Lens Studio 5.23.2, which the hackathon recommends for sign-in problems, but Interaction Kit v4 relies on an API that only exists in 5.24, so the project couldn't run there.
  • Test tooling compatibility. The LEAF package didn't compile on Lens Studio 5.24.1, so we unpacked it and fixed one type cast.
  • Frame rate in a busy Preview. At 5–9 fps, the 1-second hold-to-recalibrate stretched to ~5 seconds, so we re-timed it in real seconds. Our cue-timing test also had to allow for one frame of delay.
  • Recording the demo. Lens Studio's recorder saved six takes as empty 0-byte files ("Can't set input media type for sink"), and the Snipping Tool also crashed before we got a clean recording.
  • Designing for the glasses' field of view. Preview only draws content inside the Specs display area, so a wide step card and buttons spread across the bench got cut off. We narrowed the card, moved it next to the needle, and moved the buttons into a bar above the machine.
  • Additive display. On Specs, black is transparent, so every guide had to be bright and light-on-dark. A fine 1.6 cm seam offset may be hard to read on real glasses, which we haven't been able to check yet.

Accomplishments that we're proud of

  • Every done criterion from our design brief is covered by an automated LEAF test, and all pass.
  • One anchor for everything: move the machine, say "recalibrate", and every guide lands back on the needle (drift measured at 0.00 cm).
  • A true real-scale 5/8" seam line, measured at 1.5875 cm from the needle.
  • A Sew step that runs itself: cues fire in order with no manual input, so your hands never leave the fabric.
  • Beginner-friendly details from real sewing: handwheel and take-up lever prep, the thread hook, the "U" guide, leaving slack, and lowering the presser foot.

What we learned

  • Glasses earn their place when your hands are busy. The design started from "what can't you do with a phone here?", and that kept every feature honest.
  • Anchor everything to one point. A single needle anchor made placement, recalibration and testing far simpler.
  • Design for an additive display: bright, light content, no dark panels and small text kept close to the work.
  • Separate "logic proven" from "feel unproven." Tests and Preview prove the flow works. Comfort over time, real hand tracking and real recognition still need real glasses.
  • Test against real conditions. A slow preview exposed timing bugs that would have hidden on fast hardware.

What's next for SeamLens

Lesson 1 is deliberately one complete moment, done well. Every future lesson would reuse the same anchored-guide pattern (pinned to a different part of the fabric or machine) rather than needing a new design:

  • Pattern cutting guides for laying out pattern pieces on fabric and following the cut line with shears.
  • Curves the seam guide curves with the fabric instead of running straight, with cues to turn the wheel slower on a curve.
  • Turning corners a cue to stop with the needle down, pivot the fabric, and realign the edge to the seam line before continuing.
  • Hems guides for folding and pressing a hem allowance, then stitching close to the folded edge.
  • Bobbin winding — guides for threading the bobbin winder and loading the bobbin case (Lesson 1 assumes a pre-wound bobbin).
  • Buttons placement marks for a button and buttonhole, plus a stitch-count cue.
  • Zippers alignment guides for centering a zipper under the fabric and stitching an even distance from the teeth.
  • Pockets placement and stitch-line guides for a patch or inseam pocket.
  • Run it on real Specs, and tune text size, guide brightness and the pause button's placement for comfort.
  • One-tap needle calibration on a real machine instead of placing a model.
  • Camera-based fabric-edge tracking, so Sew cues follow your actual seam instead of a simulation.
  • Machine profiles: pick your machine model so SeamLens loads its exact thread path, bobbin type (drop-in or front-loading) and unwind direction. Popular models (Brother, Singer, Janome) first.

Built With

Share this project:

Updates

Submission history