Inspiration

Apple recently released the Macbook Pro with Touch Bar and I feel incredibly underwhelmed by it. The Touch Bar is simply too small and the dimensions too limiting to do anything interesting or useful.

It's a shame because touch displays have so much potential. Those with limited mobility (e.g. the elderly with arthritis) can find the traditional desktop environment painful to use. The mouse must be maneuvered with one hand into small regions and the slightest tremble will kick the mouse out of alignment.

Touch displays can accommodate larger buttons and there is no need to wrestle with a mouse — just touch. Additionally, touch displays feature intuitive controls like swipes, pinches and drags.

I wanted to use the ubiquitous smartphone as a touch panel that would make the computing experience more pleasant for casual users and more powerful for power users.

What it does

Touchable exposes certain function calls to the phone. In this version, the function calls allow one to:

  • Run any shell command;
  • Send any series of keypresses and mouse movements;
  • Send binary data (images, text etc);
  • Access and modify the clipboard.

There are very many "panels" that can be built on this library: in this hackathon, I have built the following:

  • Programmable audio mixer (useful for turning up and down the volume when you're a few rooms away);
  • Screenshot cropper: press a button to take a screenshot of your desktop, then crop it on your phone and send it right back
  • Rudimentary drawing app: for quick scribbles;
  • Custom emoji keyboard: saves time;
  • Calculator: saves time;
  • Multitouch trackpad with custom gestures: good for desktops with no trackpads, and better for the impaired;
  • Many more possibilities.

How I built it

I decided to use CSS and HTML as many useful widgets have already been built on them. They are ideal for creating platform-agnostic UI panels that work on all sorts of phones.

What's next for Touchable

Some of the features of Touchable only work on Linux right now, so the way forward is clear. I want to make it easier for non-programmers to build custom "panels" so I will clean up the code and write modular components.

Built With

Share this project:

Updates