Inspiration
We wanted to inspire youth to learn how code and robots work, as well as make it accessible to everyone with the voice recognition feature and by proving that robots can be built with cheap materials. We wanted the youth of today to have an impactful childhood by learning the fundamental knowledge of computing by commanding a robot.
What it does
Allow users to generate code based on voice commands for any simple robot. In this demo, a robot car is used but the AI is able to figure out how to write code for any simple robot based on clear and concise instructions.
This helps beginners learn and understand computer science. Complex and ambiguous commands will fail to generate working code and thus teach the user how to think like a programmer.
How we built it
We created a front-end UI using react with the material UI framework. A voice-to-text AI is used to allow users to give vocal commands in the prompt box. This is connected to a backend web server that is exposed to an endpoint to communicate with to receive the prompt. This backend webserver sends the prompt data and robot-related data to another web server that contains API keys in order to use the make calls to the code-generating AI. After generating Arduino instructions, the API web server sends a response back to the original web server, and the Arduino instructions are automatically compiled and flashed onto the microcontroller for the robot.
Additional info: The front end was created using React. The web page was split into 2 sections: the Title Section, and the Prompt Section. The Title Section consists of the title of the web page. The Prompt Section consists of the text input box and speech recognition input. This section also sends the user's input to the back end.
Challenges we ran into
Building the hardware portion of our robot was pretty challenging as we did not have much experience using hardware. We reflected back on the experiential activities we did in our engineering labs, did a lot of research on motors and Arduino, spent hours collaborating with the mentors and finally figured out how to build a robot using the limited hardware we had access to.
Figuring out how to take the AI-generated code and flash it into Arduino was another challenge we ran into. It took a lot of research and trial and error to finally come up with a functioning method.
Accomplishments that we're proud of
One thing we are proud of is that we gained more experience working with hardware and implementing code to work with our hardware as this was an unfamiliar concept to us. We are also proud of the fact that we came together as a group and built an impactful project in a limited time and with limited resources.
What we learned
One thing we learned was how to set up speech recognition on a web page using the React Speech Recognition hook. This hook made setting up speech recognition surprisingly easy, as it was as simple as defining some variables and using the provided functions to start and stop when the web page will be listening to the user's microphone.
What's next for EduBot
Since the AI is able to work for a multitude of robots, i.e walking robots, helicopters, etc. our next goals are to build other various types of robot prototypes to test and improve the AI's code-generating capabilities
Log in or sign up for Devpost to join the conversation.