AIFred

• 

  AIFred Chat Window

Inspiration

The inspiration for our project was to create a Jarvis like chatbot experience. We decided to name the project AIFred as a nod to batman's assistant Alfred.

What it does

AIFred was envisioned to be text free with users interacting with AIFred using their voice only. However we ran into several hiccups with the api's that prevented us from getting to a complete point. At this point AIFred has standard chatbot features and UI, and the text to speech exists as a stand alone aspect of our project and is not yet implemented into the AIFred experience.

How we built it

We developed a basic html page that we could easily host on github pages and were beginning to make a react app, however, we quickly realized if we wanted to start utilizing the api's we would most likely be writing in python. We realized if we were already going to be writing in python we would be better off using reflex for seamless integration between the api code and the front end.

Challenges we ran into

Our biggest challenges was working with the api's and dealing with the slow hackathon wifi as we had to download the models and their necessary libraries. Every download taking around 10 minutes definitely slowed down our progress especially with our smaller team. Furthermore, the scipy library that seemed to be the go-to for writing .wav files would not work on our devices. We tried several different models and tested almost every edge case, changing the sample rate, reformatting the input to the other accepted formats, changing the size of the input, and trying out the other solutions provided on Hugging Face. Lucky, we ended up finding a solution that worked but a lot of time was burnt on this issue.

Accomplishments that we're proud of

We are pretty proud of the work that we did. We had never worked with API's before and it was one of our goals to work with them, so we are happy with what we learned and it was fun to figure out how to hide our API keys and get the models working in our code.

What we learned

We learned how to set up projects on GitHub pages, build websites in Reflex, and how to work with and integrate AI into our project with the help of API's.

What's next for AIFred

Next up we want to finish implementing the text to speech as that was almost finished but not complete and then get the speech recognition and transcription up and running.

Built With

• github
• html
• openai/chat-gbt3.5
• reflex
• sudo/bark