After growing up learning programming concepts from great tools such as Scratch, Logo and Processing, we realised the power but also the limitations of using visual feedback to make learning programming more rewarding.
Tools like these and many others, rely on visual mediums to communicate core programming ideas to those who are new to programming.
However, people with visual impairments are locked out of learning to program in this intuitive, rewarding manner.
We wanted to break down this barrier, and bring them the power of instant feedback learning techniques, vastly increasing the accessibility of programming education.
What it does
Our project provides an environment for people with visual impairments to learn core computer science concepts.
Our project contains a fully featured online editor which uses text to speech technologies to make text editing accessible by speaking the code the user is interacting with back to them in an intuitive way.
The editor allows users to program in a novel, innovative language designed specifically to provide an excellent learning experience, with no setup required. With this language users can use a fun, intuitive and beginner-friendly API to create all manner of sounds and music. This language brings the instant feedback used in visual beginner languages, such as Scratch, to the domain of audio.
Users can write code in their browser, and our custom interpreter and audio engine will compile and play whatever sounds and music their hearts desire.
How we built it
Our back-end is written in Python, it uses the TextX PEG parser generator and pydub audio manipulation library. Our online editor is written in Angular.
Challenges we ran into
It turns out that designing and implementing a language from scratch is a lot harder than we predicted. We also did not expect that the audio manipulation would be as challenging as it was. Our final major hurdle that we overcame was the streaming of the generated sound back to the users browser.
Accomplishments that we're proud of
We are very proud of our final product, and we firmly believe that it can be a very useful accessibility tool for visually impaired learners in the future.
Our language syntax was specifically designed to be as friendly to text to speech editing as possible, minimising use of special characters, and matching English speech as closely as possible. We are extremely proud of how this turned out and how readable (literally) it is.
What we learned
Each of our team members learned much about their respective areas of the project. Between language design, audio manipulation, and dev tool design we were forced to familiarise ourselves quickly with many areas of computer science in which we had no prior experience. There was a lot of co-operation and pair programming so the new knowledge was well distributed among the team.
What's next for Muse
We have great plans to continue our development of this project throughout the coming years in the hope that this project will enable as many people as possible to enter the world of computer science that otherwise may not have had the opportunity.