• 

  Our logo/Mascot built in MidJourney! It needs a cute name <3 Maybe dot?

Inspiration

For 7 years I've run an e-commerce company for people living with Type 1 Diabetes. Although millions of people live with the chronic condition, and most people have heard of it, very few understand the daily routines, product needs, conversation styles, and humor trends that exist in the T1D community.

Responding to comments on public facing social media page without this knowledge would be damaging to our brand, even catastrophic given the current cancel culture and sensitive nature the condition.

When we do respond in a clever, knowledgable way, our customers feel appreciated and understood. Non-customers are able to learn more about diabetes and the devices associated thus lessening the stigma of chronic care devices (for example our most common questions are What is that? and Does it hurt)

Examples: Comment: Does it hurt?

Educating response: It's something you get used to! People affected by diabetes are no strangers to needles and pricks/pokes. There is a needle in the applicator of the CGM, but the needle doesn't stay in your body. Only a small hair-thin filament that tests blood sugar.

Comment: I hate it when that happens, I get so frustrated ..😡

Sales response:Sounds like you need an ExpressionMed tape 😉 Have you tried them before? We have over 200 designs to choose from!! bit.ly/ExpressionMedHome

We build such a good relationship with our community through comments that our youtube videos began to go viral with videos getting over 10 thousand comments in just a few days. When we responded to all comments, the videos would spread further, and we gained thousands of customers. Sadly, our one customer service rep could not keep up. Training a new rep would take months given the specialization of the conversation.

We needed a solution that could respond to future customers and educate the community en mass. To transfer what felt like comment sense to my team to a bot.

Enter, Commentsense 🤖 Chatbot a solution for my company ExpressionMed an any other influencer or company who wants to respond to comments in a community where niche knowledge is a necessity.

What it does

The CommentSense app works as a single API that loops through the most recent comments from a YouTube video, skips ones that have already been responded to by the brand, and responds to comments on the videos / shorts using the brands knowledge, sense and tone.

How we built it

We are leveraging OpenAI’s vector embeddings and pinecone vector database for content enhanced generative question answering. We are leveraging the YouTube API to create comments based on the responses that are generated by OpenAI’s davinci-text-003 model with pinecone giving us the additional context.

The vector embeddings are based on the brand, expressionmed.com’s knowledge base and faqs. This knowledge is created into vectors and then stored in the vector database.

The pattern is the following: Create Embedding Query Vector Completions API Upsert, Retrieve and Update The Upsert function enables us to generate an embedding, save it with the text from the knowledge base in the metadata and vector array into Pinecone. The Retrieve functions allows us to pass query the vector database with the embedding based on the question, return back the text from the metadata as a context to be used in the completions API. The Update function enables us to generate an embedding, search for the corresponding vector Id in Pinecone, and update it with the text in the metadata and vector array into Pinecone. Once we have the metadata (the text from the shared knowledge base / youtube comments), we can then generate the answer, and pass back the data using the YouTube Data API. We spend a lot of time on the authentication process; not only for testing the OAuth2 flow but also for being able to respond from the Brand Account on YouTube. Name: We prompted gpt 4 with:Come up with names for an ai tool that is trained to respond to youtube comments. It is trained on a language database that is customized to the brand. It is great for smaller companies in niche markets. The third example was comment sense.

Mascot/Logo: We utilized midjourney.ai to develop our lil robot mascot [See picture and website] We tried a lot of descriptive prompts but all of the logos looked crazy or had letters that were unintelligible.

The winning prompt was not descriptive at all, and it was our favorite Winning prompt: This is uncommonsense ai chatbot logo --ar 16:16 --s 1000 --style raw - Image I would not have prompted a cute robot logo, but once I saw it I loved it!

4

Challenges we ran into

-YouTube’s Data API is not well documented / easy to leverage for Brand Accounts OAuth2 Refresh Tokens

-Focusing on the use case of responding to video comments instead of trying to add more features.

-Training the AI to be able to respond based on the brand’s knowledge base, ensuring that it responds well to unique questions. -It is 19 minutes to the deadline and we are getting a google token limit error. Hopefully we will have it running gain before the deadline. Otherwise we will show in the am! Everything breaks last minute

Accomplishments that we're proud of

We spent 5 hours trying to get past the 0Auth to work for a Merchant account because it's the only way you can write comments as a youtube account. When you transfer your account to a merchant account, it is no longer connected to your gmail. Instead it is connected to a new google plus account that cannot be logged into or used for 0Auth. This is why there are no successful GPT4 Chat bots. There were conversations about this problem on quora, reddit, and github and none of the solutions worked.

We could post AI generated comments from every email we had, but we could not post from the merchant account because it was locked by the google plus email.

We figured it out a minute before being kicked out of the venue and went home with only a google token usage issue which seemed like a much easier problem to solve.

What we learned

• Persist even if you feel like failing, almost every problem is figure-outable
• Sometimes giving the AI more creativity and making less decisions in your prompt is advantageous. Our logo prompt had almost no guidance regarding the visual outcome.

What's next for Commentsense.ai

We are going to test this on ExpressionMed’s youtube account (we will show and tell tomorrow)! And then on an influencers account so see if we can build out a successful comment bot for any account. Build a user interface so customers can review their comments for further adjustments Build the caption and the transcription into context for answering questions on a video by video bases Training comment & email tools to specific knowledge bases to increase personalization. I.e. writing emails to moms and athletes. Each demo with their own trained knowledge base specific to the company, product, and community.

Built With

• javascript
• next.js
• node.js
• openai
• pinecone
• vector
• vector-embeddings
• youtube