Observe Plugin

Do you integrate LLM chatbots with your users? Do you see a decrease in sales or engagement with your LLM chatbots? Understand where your LLMs' responses are going off the rails with our plugin.

Comment 2

Inspiration

Desire to use simple natural language interfaces to understand why our LLMs are not giving good responses

What it does

Allows you to load in a dataset from a URL and ask questions such as cluster the data into meaningful groups, ask it what these groups represent, see which groups have positive/negative sentiment, and even graph these clusters into a 2D plot.

How we built it

GPT-4 for summarization, HuggingFace for embedding generation, umap-learn & hdbscan for data science, fastAPI for api build, and chatGPT plugins.

Challenges we ran into

Chat context is still too small to be able to load multiple pieces of information from large datasets to ask granular questions, concurrency issues to be able to compute backend data science API calls at the same time

Accomplishments that we're proud of

working demo that shows the possibility of easy to use model observability tools so that you can fix issues in your chatbot responses and product recommendations faster

What we learned

We can speed up the pace of data exploration, quickly and accurately asses chatbot performance

What's next for Observe Plugin

Integrate into company tools to be able to quickly load and analyze these datasets

Built With

  • chatgpt
  • gpt-4
Share this project:

Updates