EMOTI-CUE

• 

  Image Processing Steps

• 

  Emotion Detection

Inspiration

Malcolm Gladwell's book, Blink, discussed surprisingly that good communication can reduce the number of malpractice lawsuits against a doctor. It can even be used to predict which doctors are likely to get sued. We want to be able reduce this.

What it does

We built an app that can monitor and analyze the video conversation stream between a doctor and their patient and give feedback on how the patient was feeling in terms of emotions.

How we built it

• iOS
• Apple ML Kit
• Python
• Google Cloud
• Image Processing
• Machine Learning

Challenges we ran into

• Streaming data from mobile to cloud services.
• Collecting enough data to train computer vision models.
• Learning computer vision.

Accomplishments that we're proud of

• Created an iOS prototype in the time limit.
• Trained a computer vision model.

What we learned

• Learned computer vision model workflow.
• Learned iOS
• Learned how to send an image to a REST service.

What's next for EMOTI-CUE

1. Handle more complex emotions
2. Implement streaming from phone to cloud service
3. Session analytics
4. Create a generic platform to provide QA to different industries that use video streaming
5. Parameter tuning

Built With

• apple-ml-kit
• computer-vision
• google-cloud
• python
• swift