Mnemonic

Inspiration

We meet hundreds of people throughout our lifetimes, from chance encounters at cafes, to teammates at school, to lifelong friends. With so many people bumping into and out of our lives, it may be hard to keep track of them all. Individuals with social disabilities, especially those who have a difficult time with social cues, may find it a challenge to recall any information about the background, or even facial features, of those they met in the past. Mnemonic hopes to provide a seamless platform that keeps track of all the people a user meets, along with details of how they met and the topics discussed in previous encounters.

What it does

Mnemonic is a social memory companion that uses IBM Watson's AlchemyLanguage API and Microsoft Cognitive Service API to keep track of all the people a user meets. Along with date and location information about each first encounter, Mnemonic uses natural language processing to find relevant keywords, which serve as mnemonics. This social platform can be useful not only as a networking tool, but also as an accessibility platform that allows individuals with social disabilities to interact with others in a more seamless way.

How we built it

There are three main components of Mnemonic: an iOS app, a Linode server, and a Raspberry Pi. When a user first meets someone, the user triggers the start of the process by pushing a button on a breadboard. This push triggers the camera of the Raspberry Pi to take three photos of the person that is met. This also sends information to the server. The iOS app constantly sends post requests to the Linode server to see whether an action is required. If one is, it either matches the photos to an existing profile using Microsoft Cognitive Services Face API, in which case the app will pull up the existing profile, or it will create a new profile by recording the audio of the conversation. Using IBM Watson's AlchemyLanguage API, we analyze this data for any relevant keywords, using these as keyword topics to be stored in that person's profile. The user can use this information to more easily recall the person the next time that person is seen again.

Built With

• bluemix
• computer-vision
• ibm-watson
• ios
• linode
• microsoft
• python
• raspberry-pi
• swift