Ask the bot where you want to go and choose according to mood.
The bot replies with all the possible options.
Submit an image and translate to desired language
Submit a video to summarize
Send tweet directly from bot.
Lead capturing for businesses.
We wanted to build a bot that allows multiple users to participate simultaneously in the travel planning process and allow them to find the ideal destination based on their mood and available time. We knew that a major problem that travellers including us face is foreign language sign boards and long descriptive videos.
What it does
It helps users find the ideal destination based on the available time and their mood by chatting with it and filtering responses through chat messages. The user can also click any picture and send it to the bot from which text is extracted and then translated to the desired language so that they never get lost. Plus nobody wants to watch those long travel guides, right ? We allow users to upload videos and we then convert the speech into text, feed that text to our summarizer and give them a short summary of the entire video along with sentiment analysis of the summary. The user can also reach out to nearby resources by calling them directly while also allowing them to tweet about their experiences directly from the bot.
How we built it
We used Node.js for the basic chat functionality and binding callbacks. We used HPE API for OCR, Speech analysis and Sentiment analysis API, here's places API, Twitter's REST API and Yandex's translation API for achieving different functionalities. The OCR API allows us to extract text from images which is then fed to Yandex's translation API and the translated response is then given to user. Similarly, we used speech analysis API to convert the entire audio into text which is then summarized using our own summarizer and then analysed using sentiment analysis API to check the polarity of the mentioned places in the video. We use Twitter's REST API so that guy can tweet directly from the bot and businesses can look for leads using these tweets. We are also using the Exotel's API so that the guy receives texts from nearby places.
Challenges we ran into
We used so many APIs that it became kind of hard to integrate them all together but it provided us with a lot of functionality.
Accomplishments that we're proud of
We were able to pull of a full fledged working product with all the features working as described.
What's next for Trippy
Will try and add medication functionality so that people can tell it it's symptoms and it can suggest them medicines and chemists nearby.