Team88_8H_Connec.GPT

The project aims to provide a human touch to chatbots. This is done so by having a humanoid model with voice in the chat interface. The user can prompt the chat via voice or text.

Comment

Inspiration

We were inspired by the Talking Tom Cat game and wanted to add a human touch to a LLM by adding audio and visual elements into the interactions with users.

What it does

Connec.GPT receives a prompt via voice or text and will provide a reply both in text and audio.

How we built it

We used React.js as the frontend and babylon.js to render the humanoid. We made API calls to the OpenAI API to process the prompts and the GCP Voice API for the audio response.

Challenges we ran into

We faced difficulties in figuring out how to make valid OpenAI API calls. We tried to use the Assistant's API but could not figure out the documentation. We were also limited by the amount of time we had, so we switched to directly prompting the LLM.

We tried to fine-tune an open source model like Mistral using QLORA but were limited by time and VRAM.

We could not deploy the model onto vercel and we plan that with additional time, we will deploy this as a service.

Accomplishments that we're proud of

We managed to build something cool and fun.

What we learned

Working with new people from various disciplines with a different tech stack.

What's next for Connec.GPT

We were thinking of connecting people with similar interests or personalities together via the chatbot. This could be done through a profile matching algorithm to match people's personality summary generated backend,

Built With

  • babylon.js
  • gcp
  • openai
  • react.js
Share this project:

Updates