Analyzing people’s responses to images and drawings is a technique that has been used in the field of psychotherapeutic research since the early 1900s.

Drawing is one of the ways we have of expressing our innermost self, feelings, what goes on in our heads more subconsciously like no other activity.

Behind every picture, there’s a story.

Like no other activity, drawing allows us to express emotions, experience autonomy and build confidence without the usual filters.

So, why don't we examine those drawings with a simple game? I think we have a lot to learn from them. They may give us important insights into many physical, emotional and, cognitive aspects of ourselves.

What it does

The user has 10 seconds to draw a simple picture on a blank canvas. CreativeAI tries to guess the name of the picture (house, tree etc.). The user can teach the system when it fails to identify the picture.

How I built it

The core of the application is an AWS Aurora Postgresql database with a PostGIS extension. Three main APIs trigger frequent INSERT and SELECT statements. The APIs are built with NodeJs, AWS Lambda and API Gateway.

The search API, used to identify the draw, is the most important:

  • A client-side Javascript smoothes the drawn lines.
  • An algorithm inside the API simplifies and standardize the user's scribble data.
  • A PostGIS statement plays a major part in identifying the picture.

The "stats page" queries the database returning various statistical data. A bastion host is available to run additional custom queries like a similarity learning task used to optimize the system's algorithms.

Challenge I ran into

Main challenges were around picture identification and performances.

After multiple benchmarks, I've decided to use AWS Aurora PostgreSQL with the PostGIS extension. Aurora delivers up to 3X higher throughput than a standard PostgreSQL so, was not even necessary to use a cache layer between the API and the DB. The responses are very fast, sometimes it takes less than 20ms to identify a picture. To keep the performances on top, I've decided to have database clusters in 2 different regions using the Route53 latency based route to process the requests.

An asynchronous process runs every hour to reconcile the training data between the 2 regional databases since Aurora PostgreSQL doesn't support cross-region replica yet. Wasn't a problem for this application since works well with the eventual consistency model.

What's next for Creative Draw AI

Improve the algorithm, release the data as open data or allow users to run statement via a public API.

Built With

Share this project: