Vocal Portfolio is inspired in the idea of helping creators of the Art industry develop and share their own portfolios in a quick & easy manner with the use of their voice in hardware and a web application!
What it does
Vocal Portfolio starts with a conversation with the user through a device with the Google Voice Assistant to create a portfolio. The gadget greets the user, ask for their type of portfolio, and how many images they want to showcase. The device receives all the user input and saves it in a numeric code. This code is then given to the artist so he can upload his pictures, write a title, and input their social media on the web application. After that, the portfolio is ready to be shared with anyone across the web with the link.
In addition to that, the artist can download the HTML page and add his own designs and ideas.
How I built it
Vocal Portfolio is built on a combination of hardware and software hosted on Google Cloud. The Google Nest Mini, or any other IoT device powered by Google Assistant, has access to this tool. Vocal Portfolio uses Dialogflow and actions by Google to make the conversation flow with the user. This is done by creating Intents and setting up words that trigger the conversation through Dialogflow. To process all the user input and save data to the server, a webhook is necessary. To make this work, Vocal Portfolio has a Google Cloud Function written in Node.js that handles all of the data and send a code back to the user.
The web application that receives the code from the user is written in Python (Flask), and it's hosted on Google App Engine. The application uses Google Datastore to store all the information and Google Cloud Storage to save all the images uploaded by the user.
Challenges I ran into
Since this was my first time using Dialogflow and Google actions, I had plenty of challenges while attempting to build a conversation flow. The parameters were not being saved from one conversation to the other, so this required some debugging and documentation digging. After fixing that, I jumped to the development of the webhook. The webhook receives a request but it also needs to provide a response back. This was a major problem because I was dealing with external requests and the webhook would send a response back without waiting for the external request. I spent a long time trying to fix the issue and then I decided to delete the external function and built an equivalent internally. As this was not enough, I had some problems trying to send the upload of multiple files to Google Cloud Storage.
Accomplishments that I'm proud of
This was my first project using a Smart Home device with voice functionality. Therefore, I'm proud of completing a full application that combines hardware and software on the cloud. In addition, I'm proud for developing an impactful solution for artists and people in the creative industry.
What I learned
I learned how to make actions, intents, and conversation flows on a Google Assistant powered gadget. In addition to that, I learned how to integrate a webhook on Google Cloud Functions that allows me to manipulate and save all the data from the user.