Inspiration

Loneliness. Loneliness is more than just a fleeting emotion; it's a pervasive issue affecting millions of people globally. The impact of loneliness on mental health is profound. Studies have shown that prolonged feelings of isolation can lead to increased stress, anxiety, and depression. It's a silent struggle that many individuals face, affecting not only their emotional well-being but also their physical health. In a world dominated by technology, we often find ourselves surrounded by devices but lacking genuine human connection.

What it does

This brings us to the genesis of our project. Recognizing the profound impact of loneliness on mental health, we developed Rhea with a mission – to be more than just an AI assistant. Rhea is designed not only to provide information and assistance but also to be a companion, a virtual friend ready to engage in conversations and alleviate the burden of solitude.

How we built it

Voice Input Capture:

Upon logging in, users encounter a mic button, initiating the voice interaction process. Clicking the mic button triggers the recording of the user's audio input.

Base64 Conversion: The recorded audio is converted into a Base64-encoded string. This encoding facilitates efficient data transfer and processing.

Transcription API Integration: The Base64-encoded audio string is sent to a Transcription API, responsible for converting spoken words into text. The API returns a transcribed JSON containing the text representation of the spoken input.

Communication with AI Model: The transcribed JSON is then sent to an AI model specially designed for natural language understanding. The model processes the transcribed text and generates a thoughtful and contextually relevant response.

WAV File Creation: Simultaneously, the transcribed text is converted back into a WAV audio file, capturing the model's response in audio form.

Background Audio Playback: The generated WAV file is played back automatically in the background, allowing users to hear Rhea's response without requiring additional user actions.

Challenges we ran into

There were many challenges, but the most important and frustrating of them all was Transcription. This thing took us legitimately 8 hours to fix and debug. Other than that there were also things like the 3D models not rendering properly, them glitching out and some other stuff.

Accomplishments that we're proud of

The 3d models, transcription and the response generation from the custom model.

What we learned

We learnt many new things:- 3D modelling, Three.js, git version control, decoding base64 format, authorization & authentication, team collaboration & efforts, how to pull an all nighter.

What's next for Realistic Human Emulation Assistant (R.H.E.A)

Putting a more realistic voice on her, optimizing the model, trying to get a faster response, adding Facial Expressions to her face, more gestures, Emotion Detection to better detect the user's mood. Oh and also, making the application more responsive.