As a developer who writes code in various programming languages, sometimes I forget how to write syntax in a particular language, I end up reading through documentation or searching Google in order to find the code I need, but doing that slows down development time. What if there was a way to quickly insert forgotten code in your code editor without having to research? and what if you could simply speak to your code editor and tell it which code to insert? that's were Co-Pilot comes in!

What it does

Co-Pilot is an AI voice assistant that aims to help developers speed up software development time by simply speaking to their code editor in natural language. With Co-Pilot, you can write code with natural speech commands such as "add a for loop" or "insert a try-catch block" rather than memorizing hundreds of hotkeys and syntax. Co-Pilot helps you save time, Instead of typing long lines of code you can simply generate the code with natural speech in no time.

How I built it

Natural language: I used for natural language processing by adding a few sample utterances that the user might say such as:

  • Insert a try-catch block
  • Add a switch statement
  • Show me how to write comments in Javascript?

I then created a custom intent and labeled the entities in the utterances.

Frontend: I used React to build the web application and added the speech recognition API. Using the API I make an HTTP request and send the spoken message which returns an intent and entities. I make another HTTP request to AWS which sends the intent and entities to the backend.

Backend: In the backend, I used Dynamo DB to store and fetch information via API-Gateway. The code stored in the database is fetched from Mozilla JavaScript Reference.The the database is then queried to find the correct code based on the intent and entities received from the frontend. The diagram below illustrates the process: Alt text

The client records their voice in the web application, the information is sent to, which returns an intent and entities which are then sent to the backend.

Challenges I ran into

Finding the correct way of sending voice data to with JavaScript was a challenge.

Accomplishments that I'm proud of

Building a tool that solves a problem I face, sometimes I spend months working in Java or python then when its time to work in Javascript I end up forgetting the JS syntax. I hope other developers find this helpful.

What I learned

I learned how to build with

What's next for Co-Pilot

Next, I plan to create extensions of Co-Pilot for VSCode and other code editors.

+ 4 more
Share this project: