KawaiiFriend

Meet KawaiiFriend: An AI anime companion powered by Python and OpenAI used to promote both social and cultural equity.

Comment

Inspiration

I drew inspiration for this app when I realized how challenging it was to search the web without falling into distractions like social media. Furthermore, I noticed significant limitations to the internet that couldn't be overcome with a simple Google search. This led me to envision a delightful anime character that could engage with me using the charming language of anime. In today's world, accessing mental health resources remains a struggle, so a downloadable desktop application could potentially revolutionize one's lifestyle.

What it does

The app serves as a desktop assistant, offering users a chance to interact with an enjoyable anime character that can assist in problem-solving and provide relaxation through activities like gaming. KawaiiFriend emerges as the solution for moments of loneliness and forgetfulness.

Initially, the non-anime version featured several functionalities such as setting reminders and creating to-do lists. However, this list was limited and lacked comprehensiveness. While the Whisper API served its purpose, it lacked a personalized touch and the app's capabilities were restricted.

To overcome these limitations, I developed an updated version of the app that leverages ChatGPT, resulting in a more personable AI assistant.

How we built it

I employed the OpenAI API for backend functionality through API calls. Additionally, the Twilio API enabled users to make phone calls from a Virtual Privacy Network Phone Number provided by Twilio, should they wish to do so. The app was embedded using Tkinter.py for desktop applications. This choice prioritized security, as web environments can pose risks of API key exposure. Moreover, it enhanced the AI's personalized nature. To imbue the AI with a more natural sound, I utilized VoiceVox, mitigating the robotic monotony.

Initially, I experimented with pyttsx3 for audio transcription, but encountered issues such as transcription errors. Ultimately, I shifted to a text-to-speech approach using Librosa.

Challenges we ran into

The VoiceVox API and software were in Japanese, posing a language barrier. Overcoming this challenge required me to dedicate two hours to relearn Japanese to understand the documentation. However, due to its Japanese orientation, the speech output didn't align seamlessly with English words. As a result, I pivoted the project to focus on a "weeaboo" desktop assistant with anime aesthetics.

While attempting an AR model with Echo3D, my artistic skills fell short, leading me to abandon the effort around 3 am. A persistent challenge was depleting credits in my OpenAI account, prompting repeated adjustments to my credit limit. Additionally, Twilio calls in my web app encountered glitches that were resolved after delving into Twilio's documentation and learning about specific virtual privacy phone formatting requirements.

Although the frontend could have been more elaborate, significant backend improvements compelled me to conclude my efforts.

Accomplishments that we're proud of

My unfamiliarity with the GPT API, AR/VR model creation, and even Python programming before this project made my progress pleasantly surprising.

What we learned

I gained proficiency in utilizing the GPT API, a significant advancement from my prior knowledge. I also grasped the complexities of AR/VR development, realizing it's a lengthier process that often employs Unity C# instead of Python. Additionally, effective time management became evident through the value of all-nighters, as they facilitated my journey into AI and NLP app development.

What's next for KawaiiFriend

While the frontend fell short of my aspirations, I would have aimed for an app integrating VR, had I more time and a Meta Quest device. Moreover, the language barrier limitation of the VoiceVox API could have been addressed with a translation API for subtitles.

Should the product reach completion, I plan to donate any revenue generated to charity. Originally a passion project, I intend to maintain that spirit. As an avid anime watcher and gym enthusiast, these past few days provided a unique blend of both activities.

Stay tuned for the full video release of "KawaiiFriend" soon.

Requirements: OpenAI API Key, VoiceVox API Key, Twilio API Key, tkinter installed, and Librosa installed.

Built With

Share this project:

Updates