Customer Service Bot

Inspiration

We have been following OpenAI's GPT-3.0 API capabilities, but after a long period of testing, we found that its ability is not satisfactory. The release of GPT-3.5 API in March 2023 is exciting, and we can now develop excellent products similar to ChatGPT. For LLM, our first idea is to turn it into a website customer service, which is the most direct and suitable solution for LLM scenarios. We have previously opened a store on Shopify and found that answering customer questions in real-time can be a laborious task. Currently available chatbot solutions are either too expensive or require extensive configuration. There is room for improvement.

What it does

After Shopify merchants install our app, our app will automatically learn the store's existing store policies and products (including prices, descriptions, suitable users, application scenarios, etc.). We allow merchants to set up and prompt the customer service robot, including telling the customer service robot the store's purpose, how to respond to customers, and when to give discounts to attract customers to place orders. Merchants can also set the name, logo and other settings for the robot. After the setup is complete, a floating window will appear on any page of the merchant's website. Users can click on the floating window to enter a conversation with the robot if they have any questions on any page. With the support of OpenAI GPT-3.5, the robot can reply to any message very quickly. GPT-3 will respond very well to common sense questions, and it also has the ability to calm guests' emotions. Since GPT-3 has multilingual support, the robot can directly reply in the corresponding language when communicating in most common languages (such as English, German, Arabic, French, etc.).

How we built it

We first designed version 1.0 (launched on April 28th), which required developing and integrating three major system components: 1) a robot module that can communicate normally on the website, 2) integration with the Shopify management interface, and 3) LLM debugging and GPT3.5 API calling. After about three weeks of testing, we successfully fine-tuned the prompt and can provide personalized recommendations based on the store's settings. We passed Shopify's approval and went live in the app store.

Based on customer feedback and defects from version 1.0, we designed version 1.1. We allow customers to customize the robot's logo, and we embedded Pinecone's vector database to handle large amounts of text-based content, such as store policies, and embed the customer's entire store products into the vector database, supporting up to 20,000 items/products/store for learning and recommendation.

Based on our understanding of the product and customer feedback, we designed and launched version 2.0. We fully utilized Pinecone's capabilities and designed a knowledge base feature to allow users to customize customer service to a greater extent. We also allow customers to customize the LLM model that customer service uses, currently supporting GPT3.5, GPT4, and Claude models.

Challenges we ran into

1.To achieve an interactive chatbot for the storefront, the chat interface itself is a major challenge. How to ensure that it does not affect the loading speed of the store, while minimizing the server load. The difficulty lies in the fact that our team only has 1 Java developer, who doesn't have the ability to develop such a product.

2.The context limit of gpt3.5 is only 4k. We have a huge amount of store policies and product content that needs to be included in the context in order to meet our requirements. However, this approach may not be feasible.

3.We have never used a vector database and have not developed a recommendation system.

4.Customers can to some extent train large language models. However, how to prevent particularly knowledgeable customers (competitors) from hacking the model and leaking information that should not be disclosed, such as store information.

Accomplishments that we're proud of

Our customer payment rate exceeds 10%. Every day, our customer service chatbot needs to handle over tens of thousands of customer requests, communicate with customers, and process their needs. Our customers come from all over the world, including the United States, India, Germany, Bulgaria, Italy, Iceland, and many others.

What we learned

How to Use Vector Databases,There are many options out there, but we chose Pinecone.

What's next for Customer Service Bot

We will be able to handle the most common question "where is my order". We will help more global merchants to address customer inquiries and increase store sales.

Built With

• amazon-web-services
• claude
• gpt3.5
• java
• pinecone
• python