whisperChat

Smarter conversations start here!Your voice AI assistant using your own data to improve cross-functional collabortion, increase team productivity and reduce team friction.

Comment

Inspiration

This project was the ideal blend of passion to build an intuitive, efficient, and advanced AI product that can improve productivity and the fact that we've all been in situations where meetings lasted for hours. In these meetings, it was clear that people just didn't understand each other, especially when they had very different domains of expertise.

What it does

WhisperChat is an advanced voice assistant for efficient communication, personalized context-aware interaction, persistent memory, and tailored data interpretation. Features include:

  • Fully voiced, hands free interaction, using the OpenAI whisper API for speech to text, as well as the Google cloud API text to speech.
  • Persistent, multi user memory, with the ability to injest and interact with custom documents.
  • The ability to adjust its persona and response type based on user profile and professional aptitude.

How we built it

  • Pinecone - Persistent memory, knowledge-base support & user tailored response based on background. & experience
  • Docker - Containerization and deployment.
  • ReactJs - Web Framework for frontend service.
  • NodeJs - Server Environment for backend and pinecone services.
  • OpenAI Whisper API - ChatGPT and Whisper model integration for chatbot functionality.
  • Google TTS - Converts text into natural-sounding speech in a variety of languages and voices.
  • Whisper Hook by chengsokdara - React Hook for OpenAI Whisper API with speech recorder, real-time transcription and silence removal functionality.
  • Langchain - Framework for developing applications powered by language models.
  • AWS - Deployment & Infrastructure

Challenges we ran into

  • Learning new technologies and API's.
  • Understanding intuitive user interactions with AI conversational Assistants.
  • Working with a multidisciplinary team across the globe in 3 timezones.

Accomplishments that we're proud of

  • WhisperChat includes a large number of features synergising well together.
  • Code Modularity using best standards & practices.
  • Simple, intuitive UI.
  • Built in memory, using advanced techniques to improve performance and reduce cost, thanks to Pinecone & Langchain.

What we learned

Several Bleeding edge technologies and API's, including:

  • Pinecone
  • OpenAI
  • Langchain
  • Prompt Engineering
  • Working in a multi-disciplinary team is a challenging, but rewarding experience

What's next for whisperChat

  • Further enhancement of built in memory, via Pinecone.
  • Improvements in customized response, based on User background and experience.
  • Chat Assistant customization, including a profile and a customizable voice.
  • Taking advantage of Bidirectional Audio Technology for more natural, streamlined conversations.
  • UI Revamp
  • Establishment of a production-ready, robust AWS deployment infrastructure

Built With

Share this project:

Updates