We realized that not everyone has the time to learn how to code and, in some cases, not even the opportunity. We wanted to create something that would allow anybody to program and use the power of modern computing to improve their day-to-day lives. We believe that this is a necessity that everyone should and realistically can have. To that effect, we decided to make programming one step easier by abstracting away the challenging syntax of programming languages. Rather, we allow for plain english commands to run even the most complex of programs and functions. Peacock is a new programming language with an online interface accessible to anyone and everyone around the world. Our vision for this product is to introduce a new level in the hierarchy of high-level programming languages. With a language like Python, we saw amazing leaps made with respect to readable and understandable syntax, but we want to push it further. Join us in our journey to make programming part of everyone’s everyday life!
What it does
Peacock is a new programming language. Peacock allows the user to issue a command in grammatically correct plain English and instantly receive the result of that command. For example, if you’ve ever needed to sum a range of odd and prime numbers from 1 to 10,000 (and who hasn’t) we’ve got you covered. If you’ve ever needed to quickly scrape large databases of information relating to, say, stock trading, or Bitcoin, you’re now just one speech command away. Many other mind-blowing use-cases can be seen in our demo and by visiting our site. For the visually impaired, we allow for verbal commands which then get parsed into the corresponding programmatic commands and the output is returned in mere seconds, or even milli-seconds. Thanks to Peacock, we hope to be able to make programming accessible to anyone around the world.
How we built it
Parser (Natural Language Processing and Context Free Grammar Analysis): When Peacock receives instructions, it parses them into individual commands, which are then translated into Python commands, which are then executed. We used various NLP methods such as Topic Modelling, Aspect Mining, and Named Entity Recognition to appropriately categorize and identify major topic areas and group similar commands together. Once creating the proper groups/categories, we applied various Context-Free Grammar rules to appropriately dissect and extract the key information from each English command. This allowed us to formulate the exact Python execution for the user command. In that way, we are able to return the correct output in a manner that is time-efficient and highly accurate. By figuring out the appropriate key-information for the commands, we effectively close the gap between the ambiguity in the English language and the rigid structure of a programming language such as Python.
Translation/Execution (API Usage, Python Compiler): Once the key information from the English command is extracted, the next step is to translate the commands into the actual Python code. To perform this, the main function/command was determined for the given text. Then using part-of-speech tagging, elements were converted to either function calls or arithmetic operations. These were then combined together to create the Python code. Most of the desired functionalities are a lot more complicated than executing a few python lines. As a result, many of the additional features provided by our product integrated with the Google Cloud and OpenAI APIs.
Front End (Speech Recognition, UI/UX): Our front-end is pure React (hosted in a public bucket in Google Cloud Storage) and uses the Fetch API to integrate with our backend, which is hosted on Google Cloud App Engine. We use the SpeechRecognition webkit to allow commands to be submitted by voice, the most natural form of language.The interface is clean, neat and simple, and adds no more complexity than is necessary; we simply take in a command and display the result. This stays true to the spirit of Peacock, which exists to provide a simple, natural, and accessible means for anybody to code.
Challenges we ran into
Being able to effectively parse the commands and run them as Python instructions was very tricky due to the intricacies of the English language. After a lot of research and surfing through academic papers that tackled this very problem, we came up with a working solution of which we are extremely proud. Although there is a lot of room for improvement, we were able to cross a majority of hurdles related to the ambiguity of the English language. Another challenging aspect of this project was the fact that all of our team specialize in the backend and Machine Learning side of things, so having to manage all the stages of the software development life cycle, including the development of a clean and intuitive user interface, was quite challenging. In fact, our front-end dev had precisely zero experience with React prior to this project. But with the help of the mentors and each other, we overcame these obstacles.
Accomplishments that we're proud of
We are very proud of how seamlessly we were able take in speech/text commands from a webpage and almost instantaneously return the output back to the user; it feels like magic. Peacock looks like the most simple website in the world (as do many well-designed websites, like Google ;)); however, building the application from the ground up made us appreciate the level of detail and hard work that goes into making such seemingly “simple” applications. We are also very proud of how well we were able to coordinate with one another, despite the fact that it was 5am in the morning after an all-nighter.
What we learned
We learned about new technologies. We were not familiar with React, OpenAI, Google Cloud or many of the APIs that we used, but we certainly are now. We also realized how tough it would be to create a fully text-based parsing language such as Peacock. Another cool aspect of this project was learning about how python actually compiles string-based text into machine-compiler code. Using that as inspiration, we developed much of our software application in a very similar fashion.
What's next for Peacock
The next steps are to add more commands/functionality. We made the code base very modular and extendable, so as we develop this idea further, we would simply add more functionality to our API. An obvious next step is the ability to perform more complex, multi-stage operations in which outputs of one stage are consumed by another. We could also add operations on files, like renaming a set of files according to a rule, or converting from one format to another (this functionality does exist elsewhere, but the goal is for Peacock to represent a one-stop computing shop). Our goal is to also make this an open-source project, where developers around the world could contribute their own commands. With more contributions, we can turn the English language into a programming language. We’d also like to find a better way of parsing. Currently the parsing depends mainly on verb/keyword extraction using various NLP methods. It would be helpful to change this parsing technique since it is quite time-consuming to implement and somewhat rigid.