Track 4: Smart Sales Helper

Inspiration

Every time we talk to someone in a formal setting, we are always careful with our words. If there is a tool that keeps track of what you have said and gives you prompt feedback, it will make formal conversations a lot less stressful.

What it does

The Smart Sales Helper Bot detects to-be-avoided words and the formality of language during meeting calls on the Lark app. When to-be-avoided words or informal sentences are detected, the bot will send a reminder message promptly in private chat with the user.

How we built it

We used Google speech-to-text API to transcribe speech in real-time. After transcription, informal languages are detected using our trained machine learning model, and the to-be-avoided words can be found and added in the file avoid_phrases.csv.

Challenges we ran into

• Finding an audio library: At first, I wanted to use PyAudio. However, PyAudio is not entirely compatible with the M1 chip Macbook. I've spent a few days resolving the issues. Eventually, I switch to Soundevice instead, which works just as well.
• Lark API and events: I wanted to use Lark's stream recognition API instead of Google speech-to-text for better integrations. However, I cannot use Lark's stream recognition API under the free plan. Other than that, Lark's "Join Meeting" event can only occur if the meeting is booked under OpenAI. Currently, the bot calls the "meeting started" event to start listening, which means the bot only can listen to the person who starts the meeting.
• Dataset: The dataset I use rates the formality score of sentences were obtained from news, blogs, and emails. Currently, there isn't a dataset that rates formality scores specifically for sales conversations. I had to add some new data manually and add to-be-avoided word detection to make the bot function better.

Accomplishments that we're proud of

I'm proud that the model works accurately.

What we learned

Keep trying until it works. Put your heart into it and it will work eventually. Other than that, focus on the big picture and end goals. Try different methods and libraries if something does not work as expected. I think this tremendously helped me finish this project myself in this short amount of time.

What's next for Smart Sales Helper Bot

• Currently, the bot only monitors users who start the meeting. Although the salesperson is usually the one who starts the call, ideally the bot should be able to handle other cases too.
• The datasets we use to train our model are from the news, blogs, etc. The accuracy of the model might increase if it's trained on conversational/speech datasets.
• Add a bot menu for enrolling and unenrolling for a better front-end experience.

Built With

• flask
• glove
• google-cloud-speech
• mongodb
• pandas
• pymongo
• python
• scikit-learn
• sounddevice