Ideas for interactions from:

  • but I wanted to go from the existing computer down, rather from the bottom up, and make something that was a twist on the existing desktop: Web browser, Terminal, chat apps, keyboard, windows.

What it does

Maps your Mac desktop windows onto pieces of paper + tracks a keyboard and lets you focus on whichever one is closest to the keyboard. Goal is to make something you might use day-to-day as a full computer.

How I built it

A webcam and pico projector mounted above desk + OpenCV doing basic computer vision to find all the pieces of paper and the keyboard.

Challenges I ran into

  • Reliable tracking under different light conditions.
  • Feedback effects from projected light.
  • Tracking the keyboard reliably.
  • Hooking into macOS to control window focus

Accomplishments that I'm proud of

Learning some CV stuff, simplifying the pipelines I saw online by a lot and getting better performance (binary thresholds are great), getting a surprisingly usable system.

Cool emergent things like combining pieces of paper + the side ideas I mention below.

What I learned

Some interesting side ideas here:

  • Playing with the calibrated camera is fun on its own; you can render it in place and get a cool ghost effect
  • Would be fun to use a deep learning thing to identify and compute with arbitrary objects

What's next for Computertop Desk

  • Pointing tool (laser pointer?)
  • More robust CV pipeline? Machine learning?
  • Optimizations: run stuff on GPU, cut latency down, improve throughput
  • More 'multiplayer' stuff: arbitrary rotations of pages, multiple keyboards at once

Built With

+ 6 more
Share this project: