Sentiment-U

• 

  The web interface

• 

  Connected light

Challenges we want to participate

• Logitec: Challenge #1 - Streaming & Broadcasting
• Axa challenge

Inspiration

We wanted to combine the power of NLP and the possibilities offered by the IOT to create an interactive software that is useful for the user.

What it does

Imagine you get a mail from your co-worker and start writing a reply. While you write the mail behind the scene we analyze the sentiment of your content and utilize the Hue lights to highlight the emotions of the reply. This allows you to adjust the mail and be more professional.

You record a video for youtube. We use the microphone, convert speech to text and highlight the sentiment of your sentences on the hue lights.

How we built it

Like real hackers.

The core runs in Python and is event-driven.

It also implements our own API client to control the smart lights. The keyboard backlights are controlled by our custom bridge between a command-line tool and Python.

For the sentiment, we use a pre-trained Machine Learning model.

The front-end is static but communicates in real-time with the server through a websocket.

IOT is a lot of fun - when it works

Setting up the smart lights turned out to be trickier than we thought because they required us to set up a separate local Wi-Fi network. We also struggled to get the keyboard lights working on Linux, but finally, we found a good solution.

Putting all the blocks together was especially challenging because of the time constraint, fortunately, we managed to get everything working on time.

Accomplishments that we're proud of

Combining IOT, NLP, back and front end at the same time.

What we learned

Apart from the new technical challenges, we think the most satisfying feeling for us was so to see the sentiment made visible by light. We are engineers by the hearth. Talking about the sentiment as a number was what we are used to, but to see it as a colour makes it more tangible metric.

Sentiment-U to Sentiment-Everybody

We imagined several ways how to make sentiment around you more visible. For example, you live-stream a League of Legends session and want to see what your audience is discussing in the chat. We can use another Hue light to show the sentiment of the chat history. Is the audience happy - are they angry towards you - you don't need to read the chat history constantly, focus on the game experience and react if the audience is not satisfied.

We can utilize the mouse to show the sentiment of any text you highlight.

Our goal was to have a working prototype for this hackathon, therefore we use an already trained NLP sentiment analyzer rather than building our own. It would be nice to have a robust model that actually takes the context into account (eg. writing an email to your boss vs. chatting with a friend on WhatsApp).

Built With

• flask
• iot
• javascript
• machine-learning
• natural-language-processing
• python