We were initially inspired by the many Computer Science professors at UMD who like to write out their code as they lecture instead of typing it into an editor. While some use traditional pen and paper, many others use the touchscreen and stylus interface of their tablet PCs, which allows them to quickly write up code snippets and share them digitally after lecture. This code, however, cannot be immediately executed, which means that demonstrating its functionality requires retyping it into an editor, taking away from teaching time. We thought it would be helpful to find a way to go straight from handwritten code to text that can be executed in an IDE or terminal.

Over the course of our weekend at Bitcamp, this idea grew in scope, as we realized the tool's potential use to students, developers, and anyone else who might want to commit a quick note or code snippet to text without the need for a keyboard. We envision our final product as a small tool for educators, students, and professionals who prefer handwriting their code to move from idea to implementation with ease.

What it does

The Keyless Coding application uses Google Cloud Platform's Vision API to convert handwritten input to editable text. It is designed for quick code prototyping, note taking, and coding demonstrations.

When started, the application provides a lined canvas for writing. The user can fill the whole screen with text, or add code snippets incrementally, before writing the final output to a text file. The text file can then be opened in a text editor for later review or copying into a larger project. The user can also generate multiple separate files in a single session.

How we built it

We wrote the code for this project using Java in the Eclipse IDE. The user interface is implemented using the Java Abstract Window Toolkit (AWT) and the interfacing with Google's Vision API is done with the Google Cloud Tools for Eclipse plugin.

Challenges we ran into

Originally, we planned for this project to be a plugin for Eclipse IDE, rather than a standalone application. However, after a great deal of time was spent unsuccessfully trying to integrate the Google Vision API with the Eclipse Plug-in Development Environment, we determined that there were issues with compatibility that could not be resolved this weekend.

Ultimately, we were thankful for the change though, as this sped up our development pace considerably after the change and our final product is now usable by anyone, not just Eclipse developers.

What we learned

Before this weekend, we had only basic experience with creating UIs in Java and no experience with Google's Cloud Platform. Now, after an entire weekend of hacking, we are much more confident in our ability to implement projects that use these tools and hope to make use of these skills for many projects in the future. We also learned a great deal about creating plug-ins for Eclipse using its Plug-in Development Environment, even though we ended up not using it for our final product. Additionally, we reinforced our knowledge of Java and Eclipse development.

What's next for Keyless Coding

In the future, we'd like to make refinements to the UI and add more customization options for the user.

Share this project: