We were inspired by numerous no-code tools and we were wondering if it is possible to turn voice into code. After brainstorming for more than 2 hours, we came up with the concept of using Natural Language Processing and Machine Learning to convert voice commands into code.
We wish to mention our mentor, Jesus Del Valle for giving us amazing insights and improvement ideas!
What it does
"What if you can talk informally to a computer and make it code for you?" - Yes you heard it right. 'VoCode' uses Natural Language Processing and Machine Learning to convert informal speech into code.
For an example, if you want to make a login screen in your website - just say 'VoCode, make a login screen and logic for me'. VoCode automatically creates the whole UI and back-end functionalities required for making a full stack web application.
You can also choose your preferred technology stack to output the code (only PHP and React are available as of now). VoCode does all the required coding for your need and provides you the final source files of a website in a deployable condition.
How we built it
We built VoCode as a desktop application using Flutter. VoCode works on Windows, Mac and Linux platforms.
Front end : We used Flutter along with Material UI for making the front end of the application. The below are some of the screenshots of our Desktop application :
Back end : The complete backend is developed using Python. We created a CLI application and linked it with the Flutter GUI. We made it in such a way that the python scripts execute on a shell in the background when user performs any action on the Flutter GUI.
Natural Language Processing and Machine Learning : We used NLTK library in Python for Natural Language Processing. For Machine Learning, we used Keras along with Tensorflow for automatically appending the required code into the source code based on the learning from the processed speech to text data.
Challenges we ran into
- Integrating the python backend along with Flutter desktop application was the most challenging part.
- Using shell commands in Flutter took us some time.
- Since we were new to NLTK, we consumed remarkable amount of time on implementing the Natural Language Processing Part.
Accomplishments that we're proud of
- This is one of the most technically complex projects that we've worked on till date and we are super proud to have completed it :)
- There are no existing alternatives to our project and we're happy about coming up with such a great novel idea.
- Team work and time management.
What we learned
- We learnt to integrate Flutter with Python.
- NLTK and Natural Language Processing concepts.
- We learnt alot about how speech to text libraries work.
What's next for VoCode
- Adding more tech stacks for the user to output the code (We currently support React and PHP stacks to output the code).
- Removing bugs in our desktop application.
- Training our model with a better accuracy and adding more automation.
- Adding more preferences for database.