Inspiration
Our inspiration came from the movie Interstellar and the character of TARS.
What it does
It is a functioning scaled-down replica of the character TARS from the movie Interstellar. It has a functional screen, a speaker, and 4 servos to control the movement. It can "walk", converse with you, and perform simple tasks.
How we built it
We custom trained a TensorFlow model for the activation/wake word, used Vosk for on-board speech recognition, and the Google Gemini API for the conversational character.
Challenges we ran into
There were plenty of challenges. We constantly ran into issues with connectivity with the hardware and small issues that weren't even part of the code. Thankfully, we were able to overcome the vast majority of them.
Accomplishments that we're proud of
Being able to have a physical product that actually does something makes us feel very proud and fulfilled. This year's HackKU was a blast, and we're happy to be able to walk away with many new skills.
We learned how to implement voice recognition speech to text and then integrating that with gemini to text to speech.
We successfully implemented a psuedo-PID controller using an approximate distance based methodology for steady state error.
What we learned
Most of us had never dealt with this small of a time crunch for a hardware-based project, nor had we ever trained our own machine learning models.
What's next for TARS
Adding more commands, fleshing out movement, and adding even more expressiveness to him.
Eventually... space.
Built With
- 3dprinting
- assembly-ai
- gemini
- hardware
- openai
- python
- raspberry-pi
- servo
- speech-to-text
- text-to-speech
Log in or sign up for Devpost to join the conversation.