With our project, we wanted to have a social impact. Development coordination disorder (DCD) affects 5-6% of school-aged children ( However, there is no standardized procedure for diagonising this potentially lifelong problem.

The Movement Assessment Battery for Children-Test, M-ABC Test for short, is a common test used for diagnosing DCD through exercises as seen in the following video:

Our project aims to recreate this test so that it can be performed in a standardized manner through state-of-the-art technologies in robotics and artificial intelligence.

What it does

The ABB YuMi robot assumes the role of a moderator in conducting a concise version of the M-ABC Test. Using Cognitive Services and Artificial Intelligence APIs, YuMi evaluates the performance and sentiments of the participant.

How we built it

We used Python as our main programming language.

A laptop's microphone and speakers serve as the "mouth" and "ears for YuMi. The following Python packages are used for audio capture, saving, and playback: sounddevice and soundfile.

A Logitech webcam is used to give YuMi sight. The OpenCV library is used to perform standard image analysis and processing techniques such as convolution, de-convolution, spatial filtering, segmentation, and blob detection.

Finally, the following Microsoft Azure services were used to give YuMi cognitive abilities:

  1. Bing Speech API for speech-to-text and text-to-speech.
  2. Emotion API for analyzing the behavior of the participant during each exercise.
  3. Text Analytics API for sentiment analysis during the participants final self-reflection.

Challenges we ran into

Visual capture was a significant problem for us. Moreover, the analysis performance is very sensitive to different lightings.

Working with the robot was also a challenge as we had never worked with it before and prototyping was not as flexible and quick.

Accomplishments that we're proud of

We are proud of having combined many components in hardware and software. Moreover, we were happy to work on a project that could have a potentially beneficial social impact.

What we learned

We learned how to use a lot of new APIs tools and to combine them into a single project.

What's next for jred

More tests could be added to further improve the diagnosis of motor skill problems. Improvements to the audio and image processing could also yield more robust evaluations of the participants.

Built With

Share this project: