Mr. Helpsworth, Your Robot Friend

Mr. Helpsworth is your own personal robot aid. Whatever you want him to do he will do it, if your kid needs a tutor, you need a knowledgeable kitchen aid, Mr. Helpsworth can help with anything (maybe)

Comment

Inspiration

Mr. Helpsworth was inspired by retro-future helper robots such as Codsworth in Fallout 4 and Andy the Messenger Robot in Stephen King's The Dark Tower series.

What it does

Mr. Helpsworth can help you with many day-to-day tasks! By choosing a role for him, he can help you with cooking, cleaning, and even help you with your homework. His immense knowledge and advanced artificial voice show that yesterday's future of tomorrow is today!

How we built it

Mr. Helpsworth's knowledge is reliant on the OpenAI API. The website, built in pure HTML/Python, sends the user's question to our Flask backend, which queries the OpenAI API and translates the response into audio using the elevenlabs text-to-speech API. This audio is then sent to a Python program that plays the audio and sends commands to the Arduino-based robot using the MQTT messaging protocol.

Challenges we ran into

Speech Input

Since we didn't use any frameworks for our front-end, implementing the speech turned out to be very difficult. However, we managed to achieve it in the end.

Hardware Communication

At the beginning, we could not decide how we wanted to communicate to our Arduino. Every option seemed to have its pros and cons. After some research, we discovered MQTT which is a messaging protocol that works over the internet, and it solved most of our problems.

Front-End Programming

None of us have built a website before, so we had to dive into unfamiliar territory in order to make our idea work.

Accomplishments that we're proud of

Robot and Website Design

We managed to create a great looking robot and website in the 24 hours we had, despite our relative inexperience with hardware and web development.

Technology Integration

Overall, we mainly used 4 libraries in our software: OpenAI API, elevenlabs API, MQTT, and Flask. We were able to make them all work together seamlessly.

Hardware

We were able to create a quick-to-respond hardware system for our robot using MQTT. It responds nearly immediately to Mr. Helpsworth's speech.

What we learned

Overall, we learned alot about web development, APIs, and IoT communication systems. These are skillsets that are very important in the current software development field, so this project has been great experience.

What's next for Mr. Helpsworth, Your Robot Friend

Hardware

In the future, we would like to improve Mr. Helpsworth's hardware. It would likely need to be slightly bigger so we can accommodate the electronics, and we would like to add a mic and speaker to the robot so we can talk and listen to the robot directly.

Real-Time Data

We would also like to add the capability to process questions involving real-time data, such as the current time. With just the OpenAI API this is not possible, but we unfortunately did not have time to integrate real-time data in our project today.

Share this project:

Updates