Dream Journal

The fast-paced daily life has caused fewer and fewer people keep on the habits of writing journals and diaries, and one of the most important reason is that journaling is time-consuming and boring. Have you ever wondered your journals to be animated, playful and appealing, and you don't even have to write a word for it? Dream Journal made your dream come true. We combined speech recognition, style-transfer, AI Music and emotion detection, and materialized an web app that you can animate your daily memory with photos, colors and smart music!

Inspiration

Journaling has psychological and mental benefits when done regularly. However, people don’t have enough motivation to write journal everyday. By generating compelling media included AI-based background music and style-transfered images, Dream Journal provides a fun and interactive way to help user make each moment unique and memorable.

What it does

By linking user’s social media account, Dream Journal reads user’s photo and ask user the story behind the picture. Dream Journal then use cognitive related services and deep learning model, generating compelling media that reflect user’s mind and mood, creating a interactive multimedia journal.

How we built it

Backend: Node.js (web server), MySQL (for db), WebSocket (for voice and media streaming) AI: Tensorflow + Torch (for style transfer) + Python(for music generation), and Microsoft cognitive services (for voice recognition) Frontend: Everything is handcrafted! Server: Microsoft Azure GPU Instance (N-series).

Challenges we ran into

Audio streaming and voice recognizing.
The utilization of facebook graph APIs.
Training the machine to render more compelling images.
Getting the emotion from the user input
Generating emotion based musics.

Accomplishments that we're proud of

The voice recognition system that is built with Microsoft Cognitive Services. In the front-end page all the user voice will be interpreted instantaneously, while the UI will response the user's emotion, the user can even use keywords like "next" or "finish" to control the front-end system.

What's next for Dream Journal

We want to better implement our idea by pushing it further to integrate more artificial intelligence APIs to make more compelling arts to better recap the user's journal.