Current touch screen on tablets are not sensitive enough to accurately track handwriting and hand-drawing, so most users still carry a keyboard that allows them take notes on their tablets quickly. Now with Sensel Morph, you can take accurate hand-written notes on your tablet without having to carry another device besides your keyboard. Using Sensel Morph with our SenselNotebook mobile app, you can easily take notes on your tablet. When you need a keyboard, simply attach the keyboard Overlay and use Sensel Morph as a keyboard!
What it does
SenselNotebook communicates with Sensel Morph via Bluetooth, and transfers user's handwriting and drawing onto thei mobile device. The user can use gestures to change colors, undo their changes, and save their notes into their devices.
How I built it
Bluetooth connection is not yet supported on Sensel Morph. In order to retrieve Sensel Morph outputs on mobile devices, we first wrote a python script and combined it with current Sensel Python API to convert USB port messages into Bluetooth signals. We then parse the Bluetooth signals on the mobile device, and display the drawing/handwriting the user exerts on the track pad.
The application is developed in Android Studio. We developed our own gesture pattern recognition, and modified existing BluetoothChat and Canvas application to fit our desired functionality.
Challenges I ran into
Bluetooth packages are extremely unreliable. Characters/lines are often dropped during the transmission and this makes it hard to identify gestures accurately.
Working with existing modules can be both convenient and tough. Using some existing modules, we quickly built the interface between Sensel trackpad and a mobile phone. However the hard part is ensuring their compatibility with our app.
Separating gestures and handwriting is also a very difficult task. Handwriting needs to show up on the screen in real time, but gesture requires
Accomplishments that I'm proud of
Having Bluetooth connection with Sensel is the key to our idea. Though it is not yet supported on Sensel, we were able to find a work around and make it work with our application. We also built an android app dedicated for taking notes or drawing pictures on Sensel. We have some basic functions including changing the paint color and saving the file to JPG.
What I learned
All of the above...
What's next for SenselNotebook
There are lots of improvements that can be done. Bluetooth latency is definitely an issue. There are lots of features that we thought of but did not get to implement including undoing a stroke, varying stroke width with input pressure,