Inspiration & What it does

There are many sources of inspiration that came up with this project. First of all, some of my friends ask me a lot to translate a letter or some news for them. This App will let them take a picture of the letter, and they can translate the letter, and even generate audio for the translation. Secondly, I personally having a hard time to read an article or a book in the crowded subway. This App will allow anyone to send an image, then generate audio and listen instead of reading! Moreover, some students struggle with reading long articles or search about the topic. This App will scan a picture of any article, extract English Text, categorize the text, and offer up to 30 resources to read more about this topic. Also, it can provide summary to get a brief idea about the topic. Nevertheless, all of the scanned images will be saved in a personal storage, and the user can bring his files back at any time and do another process! Finally, this App can scan forms or passports and read the information in a .txt format which can be used to fill forms faster than typing.

Handover Protocol, One Time Notification & Quick Replies

I am using Handover Protocol to connect two different Apps together in one conversation. Also, Handover protocol is used it to take control of the conversation by the main bot in some cases. For example, if the user is connected with the inbox and there is no Customer Service Representative available, the user can type a special phrase to go back to the main bot. Moreover, One Time Notification is being used to notify the users if there is a new App released or before the monthly files deletion and balance reload. The One Time Notification can be sent as a text or a template from the regular messenger conversation and this feature can be triggered only by the Admin. I implemented this by assigning a variable with the OTN token in the user data. When the One Time Notification Function is triggered, it will loop over all all the users PSIDs and if there is a token the App will send the message. Finally, Quick Replies is being used to display the file names to the user. If the user have about 28 files, it will display the first 10 Files and a View More button. When the user press View More, he will see the next files and another View More and back button. Also, Quick Replies is being used all over the conversation to provide a seamless communication and support the persistent menu.

How I built it

I build this project in JavaScript using Node.js. First, I used DynamoDB for the users table data. I designed the possible table schema with the user data. Then, I implement the File Storage by using Array List for each Category with the Files name and Map that map all names to it's path. In addition, I have used general state variable which will update the user location in the App. This general state is used to limit triggering the One Time Notification function to the Admin, and handling some possible errors as well. Also, I used some variables as counters for each feature limit. On the other hand, I used AWS Textract and other Machine Learning Modules to extract the text from the image. Moreover, I used AWS Polly to generate Audio based on the text language "Currently only 3". Then, I used APIs to translate, summarize, and categorize the text. Finally, I used an API that will read the content of the text and search for the key words and offer some resources to read more.

What I learned

I have learned that when someone has passion and love something truly from his heart, he will be able to do anything in this world. Few months ago, I wasn't able to build a webhook and didn't know anything about Node.js. Now, I feel satisfied with what I reached, but still knowledge has no limits and I aspire for more and more :)

What's next for Smart Bot

I will keep adding more Apps and translate the conversation to as many languages as I can. Also, I will add a smart helper that can communicate and answer questions using Text and Voice.

Built With

Share this project: