We took the idea of using AI powered assistants, such as x.ai and J.A.R.V.I.S., to help out with everyday tasks. Specifically, we focused on the idea of documenting conference meetings in professional settings. With a need to take minutes, almost all businesses, companies, and organizations have a designated secretary. Each individual then reviews and processes information from the meeting on their own, importing dates into their own schedules and writing important notes for personal reference. We want to streamline this process of gathering, processing and recording information, taking it from a human role to an AI function. However, we found current human/AI interaction rather clunky. Rather than actively addressing the AI, like with Siri, we decided to make Callie fully autonomous. Callie, in the background, transcribes each meeting and delivers real-time information without any prompts. Callie can host a professional environment that feels natural and invariable, making the meeting room a friendly and interactive place.
What it does
Callie documents your meetings and provides a transcript for later use. To do this, Callie uses advanced machine learning algorithms from Google Cloud Platform to organize and store each session. Furthermore, Callie dynamically interprets popular names and references, linking secondary sources for further research. This allows the instant processing, understanding, and storage of information in a conference call.
How we built it
For the backing API, we used a Ruby on Rails application deployed on AWS. The chat website is a NodeJS application deployed on Heroku. The Node app uses React for the UI and Twilio for the communication. Every few seconds, it uses Annyang to interpret speech to text and sends it to Google Cloud Platform to run Natural Language Processing on it. From there, the results are given to the Rails API.
Challenges we ran into
We ran into several challenges, both conceptual and practical. First, we had to make a clear effort to distinguish our concept from other artificial assistants such as Siri or Alexa. Second, we had lots of trouble connecting AWS to Google Cloud Platform. Instead, we decided to redeploy on GCP, but after running into more complications we reverted back to AWS. Additionally, we found that the Google Cloud Platform APIs didn’t provide as much comprehension as we wished. As a result, we had to supplement them with our own attempts at comprehension
Accomplishments that we're proud of
Starting from scratch, we are proud of finishing this application coming from different programming backgrounds and geographic locations. Each with a different skillset, we managed to make Callie work. We are happy that Callie is a user-friendly and relevant tool that can be used both in a business setting and for personal purpose with friends.
What's next for Callie
In the future, we hope to develop Callie by: Integrating custom machine learning models Adding more personal assistant features such as scheduling meetings, importing dates to calendars, and creating personal notes for each individual in the call Adding pictures, screenshots, and real-time add-ons to descriptors Introducing Callie into office software for in-person meetings Commercializing Callie and monetizing the application