Inspiration

We take our clues from the brilliantly logical human mind. We try to mimic a human child's learning behaviour over certain logical concepts and augment it with the power of natural language processing. It should be noted that to approach the problem of general logical intelligence, we must deconstruct the reality around us into small but meaningful pieces. Facebook has thought about this problem and has produced a 20 task set for testing text understanding and reasoning known as Babi. We present to you an approach that can potentially solve all 20 of the tasks.

Since the Babi dataset is bi-lingual, Jasmine speaks and understands both English and Hindi. Just like humans, she can converse in both languages simultaneously*. Jasmine can also learn other languages if you converse with her in them.

Note that state-of-the art machine learning algorithms have only solved 11/20 of these problems! [1]

[1] : https://arxiv.org/ftp/arxiv/papers/1703/1703.03939.pdf

What it does

She talks with you! Given a series of sentences depicting daily life from the real world, it can answer several logical questions about intricate temporal relationships amongst objects described. What's more? Just like a new born child, Jasmine is excited to learn new concepts about the world. Unlike it's more forgetful cousins like Siri/Cortana, she can generalize on relationships you teach her about the world and its objects. For example, read the following conversation:
You: Mary picked up the milk.
Jasmine: I don't know what milk means.
Jasmine: Can you teach me?
You: yes
Jasmine: Do I Know the Concept?
You: no
Jasmine: What is the name of the concept?
You: Thing
Jasmine: What is the action I am supposed to take?
You: add node
You: John moved to the bedroom.
You: Mary travelled to the hallway.
Jasmine: I don't know what travelled means.
Jasmine: Can you teach me?
You: yes
Jasmine: Do I Know the Concept?
You: yes
Jasmine: What is the name of the concept?
You: ChangeInLocation
You: Priya rasoi ghar gayi.
You: Sandra went back to the kitchen.
You: Where is the milk?
Jasmine: hallway

How we built it

Basically, we model the reality around us using graphs evolving over time. We then translate questions asked to us into a query that runs over the temporal graph to give 100% logically correct answer. Along the way, Jasmine learns what do words mean and how everything in the external world is just a specific instance of concepts she has learnt. Jasmine can also learn new concepts and relationships on the fly! All this code runs in your browser! Taking advantage of GPUs offered by the Google Cloud.

Potential Deployments

  • It can be combined with sentiment analysis networks to get a product that is both logical and emotional
  • It can be used in warehouses to keep track of where things are going and when
  • Possibilties are truly limitless with general intelligence!

Challenges we ran into

Algorithm design: It proved to be extremely tricky to mimic human reasoning. Software design: Since your memory is (technically speaking) an array of all the concepts you have ever learnt, Jasmine can recall concepts taught in previous runs.

Accomplishments that we're proud of

We achieve 100% accuracy on Babi task 2 of which the closest state-of-the-art competitor came close to 90%

What we learned

We learnt that conventional machine learning can be augmented by a logical digital "Mind" like Jasmine to accomplish unprecedented levels of general intelligence.

What's next for Jasmine

Currently working on an object detector using Tensorflow so that we can start answering questions about videos ;)

*You can only ask questions in english for now

Built With

Share this project:
×

Updates