The inspiration for Google Draw came about when one of our team members experienced the struggle of attempting to draw with a fractured wrist. Google Draw allows users to provide the Google Assistant with vocal direction to create sketches, construct geometric shapes and fill in colours. As hands are a key element to this creative process, it is especially useful for individuals who want to express their creative side through art but have limited hand mobility or are amputees. The Google Draw works based on a coordinate-grid system. We require from the user to select indices in the number-labeled grid to create points and join them using keywords, such as “connect” or “link”. Once enclosed shapes are produced, then users may choose to fill them in with colour. We used a NodeJS library to create our own blank canvas image that updates with every dialog instruction the user provides. Using the intents and entities for the DialogFlow AI API we were able to train the Google Assistant. Lastly, we used Firebase Server SDK and Storage to integrate it all. We hit a few bumps on the way when we were trying to find out which library would be most suitable for the creation of the canvas image, without using HTML rendering. Although the documentation of the Firebase SDK Server and Storage is very well-written, we had a tough time going about the bugs due to the limited resources online. After overcoming the difficulties and engaging with new programs, the demo is now complete and this already gives a means of self-expression similar to that of able-bodied digital artists. Additionally, we have future plans of implementing an important key feature called the “Poetic Colour Palette”. This feature allows users to verbally express how they are feeling and the Google Draw application will prompt the user with appropriate colour palettes. If the user is sad, we might offer a more blue-toned palette whereas if they are relaxed, we might offer a purple palette. We hope that our innovative attempt to provide an inclusive art-based application for this standing issue sparks your interest and ignites the passion that will boost this project to its full potential.

Built With

Share this project:

Updates