MATLAB Deep Learning, Mirrored/Rotated Digits

• 

  Final results. Incredibly accurate classification -- in high 90s.

• 

  Final results. Incredibly accurate classification -- in high 90s.

Implication for Ethical AI

*Note: mirrored images include both reflection and rotation (randomly generated, so images can also include reflection but not rotation, and the inverse) due to MATLAB augmentation, but I'm writing "mirrored" for the sake of simplicity

This program attempts to address the bias in handwritten digit data sets. The classification problem is that the program initially could not classify mirrored digits off this data set. Mirrored handwriting is a symptom of dysgraphia, characterized by disoriented writing. Researchers estimate that 5% to 20% of the population have dysgraphia; moreover, dysgraphia is prevalent in children (especially those beginning to learn to write) and brain-damaged adults with Parkinson's, tremors, and spinocerebellar degeneration. Additionally, mirrored handwriting is prevalent in elderly adults with dementia.

Training a deep-learning neural network to classify mirrored digits accurately expands the capabilities of computerized technology to accommodate people with dysgraphia. For example, this program will allow computers to scan documents correctly, grade assessments, assess transactions, and test for cognitive abilities (the program can recognize that a standard digit and a mirrored digit are different versions of the same number, flagging any prominent trends).

What it does

This program enhances the training data by simulating augmented data and creating a larger solution space. In all, the program attempts to classify mirrored images as the same number as their un-mirrored counterparts -- for instance, a mirrored 5 and a standard 5 would both be classified as "5."

How we built it

This program was developed using MATLAB Deep Learning Toolbox. The data set was sourced from the Deep Learning Toolbox, consisting of handwritten digits. The mirrored images were simulated as reflected images around the y-axis using MATLAB. The final training set utilized MATLAB augmentation to simulate flipped images both by reflection and rotation.

Challenges we ran into

Initially, I tried to rotate the image 180 degrees to restrict the solution space. This method worked all digits but 5s. The 5s were not flipped but were "upside down." Therefore, data augmentation had to include image reflections while expanding the solution space. The downside of this method was that the precision of classifying the original images (non-mirrored numbers) would decrease (though only slightly).

Accomplishments that we're proud of

The results were incredibly accurate. Check the attached screenshots. Mirrored images were classified correctly in the high 90s, while un-mirrored images were still mainly classified correctly (also in 90s).

What we learned

I learned how to use MATLAB's Deep Learning Toolbox and data augmentation. I previously used the Deep Learning Toolbox for regression tasks, but not for classification.

What's next for MATLAB Deep Learning, Mirrored Digits

Next, I would like to include a more extensive data sample to include mirrored letters. Another step is to test the images on various handwriting samples since there are many variations in writing, and digits may not be exactly "mirrored." Finally, this program classifies mirrored numbers and un-mirrored counterparts as one-and-the-same, but future iterations may classify mirrored digits as "mirrored 3," for instance, instead of just "3." This modification will help with testing for cognitive abilities.

Built With

• matlab