A novel art gallery where autistic artists' paintings can be enjoyed!
A calm environment that allows you to express yourself freely and paint your heart out!

Thousand-Words

AI enhanced Art Therapy through Virtual Reality, to alleviate autism.

Inspiration

Despite getting inspired by and lauding famous people who were born with some form of autism, like Albert Einstein and Mozart, Autism is highly stigmatised in our society. Studies suggest that people with autism are often highly visual thinkers and many reports that they “think in pictures”. Expressing feelings and ideas through images is very natural for such people and can be a welcome relief from the daily struggle to use words effectively. The traditional methods of "Special Attention" only makes them feel isolated and lonely. Art Therapy is a good approach but the people don't tend to express themselves freely due to fear of being judged and the sessions are more than they can afford.

Thousand Words

We understand that AI can never replace human's feelings, but it can enhance our abilities to express ourselves better. Our app is divided into two parts: Artistic Creation and Artist-Audience Connection. We implement visuals and audio to make the experience complete.

For artistic creation, it helps to translate the user's random drawings into a beautiful artwork, which not only help them to feel better but also encourage them to express themselves. As the saying goes: A picture is worth a "Thousand Words".

As part of the artist-audience connection, our app provides a gallery to show and appreciate other autistic artists' art pieces. Looking at a peer’s artwork offers a concrete way to “see” another person’s point of view and as the two share focus on the client’s art-making, a powerful bond can be forged without the initial need for direct, face-to-face interaction.

Art Therapy

In recent years, caregivers seeking complimentary treatments which have a broader range of options available. One such treatment is art therapy. Broadly speaking, art therapy promotes mental and emotional growth through art making. It is conducted with the aim of building life skills, addressing deficits and problem behaviours, and promoting healthy self-expression.

No man is a lonely island, Expression is out-coming of your thoughts, your internal behaviour at peak. Sometimes it gives us happiness, we cheer, we cry in sorrow, feel joy one of supreme feeling when we are in love, feel angry when aggression is there.

Thousand Words is a Virtual Reality application that helps patients to explore and express themselves using art materials and crafting attractive artwork.

How we built it

We used Unity engine to build the environments. We used the free unity asset ink painter to aid us in painting on the canvas. We used an open source GitHub repository and used a flask server to connect the VR app to the server. We performed art analysis based on the density of the colours and brush strokes used in the painting and mapped it to the emotions associated with it. The painting done in the canvas is sent over the server to have the analysis performed and it returns a neural style version of the original image, which is equivalent to the therapist helping you with your art. It also suggests improvements that could be made to the painting based on the scientific studies done by arttherapyblog.com.

Challenges we ran into

We had a hard time making the existing neural nets fast as it took over 45 seconds to return the suggestions. Hence we had to optimise it by using a faster neural net as Keras couldn't handle it and reduced the processing time to around 7 seconds. We also spent a lot of time trying to make the painting experience smoother and with lesser lag time. We also learnt how to use raycaster in unity to make the description of the arts in the gallery visible whenever the user looks at it.

Accomplishments we are proud of

We are proud that we were able to successfully complete all the features of our app in such a short time. We were also happy that all the skill sets of our team members complemented each other to make this project what it is.

What we learned

We learned how to teleport in VR to reduce motion sickness to the users when they need to physically move. Teleportation helps lesser real life motion and makes the experience sweeter.

What's next for Thousand Words

In the future, we would like to implement better art analysis to help the user get more professional feedback on his paintings. We would also like to include a scroll bar that allows the user to manage how much of his art he is willing to be modified. We could also work on trying to make the analysis feedback faster to reduce the waiting time of the user. We could also dynamically change the music frequencies based on the mood of the user.