What if you could explore, learn, and discover your daily surroundings with no extra effort?
What it does
DoraX is an assistive, iOT backpack, that supplements what you see. Users interface with our backpack using our Microsoft Skype Bot.
An automatically triggered camera is mounted on the physical backpack which is triggered every time the user initiates a conversation with BackpackBot. A snapshot of the user's surroundings is captured and analyzed to produce the latitude and longitude of the backpack's (and the user's) current location. We then built a Google maps scraper to produce reviews that people previously at this location have submitted about commercial buildings in the local area in combination with crowd sourced reviews stored in our own database. The user can explore and then submit their personal opinions of the location (which contributes to the crowd sourced data). BackpackBot then produces a list of recommendations of more attractions near by and if the user is interested, will navigate the user to that destination.
How we built it
Lots of NodeJS. (And Google Maps, Arduino, iOS, and LUIS - Microsoft's Natural Language Processor.
Challenges we ran into
We did not have access to an individually triggered camera to mount on the backpack so we hacked an iPhone to take a picture whenever a user initiates conversation on the personal mobile phone and push that image to the server.
Accomplishments that we're proud of
The entire user experience is contained in a fluid experience with BackpackBot in the form of a conversational UI. Interaction with our product is organic and doesn't require the user to download any new software (just get our hardware unit). Also, the LED display on the backpack actually produces different light patterns based on what actions the BackpackBot is performing at any given point in time.
What we learned
Lots of NodeJS. Also to work with the cutting edge Natural Language Processing introduced by Microsoft.