PromptBuddy

PromptBuddy Analytics Dashboard
PromptBuddy Home Page
PromptBuddy History Page

Inspiration

In the evolving world of technology, AI has taken the spotlight in recent years. Despite this, most users are not able to utilize AI to its fullest extent. I wanted to build an app that allows users to harness the full power of AI through better Prompt Engineering.

What it does

PromptBuddy is designed to analyze and improve the effectiveness of user prompts for LLMs. By leveraging the Gemini Web API, the app breaks the user's prompt into several key points:

Intent: What is the goal of the prompt
Ambiguity: Is the user's prompt unclear or confusing
Risks: Potential issues such as hallucination, bias, or misinterpretation
Confidence Score: How likely the prompt is to deliver desirable results

The user receives a detailed analysis with suggestions for improvements, and with a click of a button, can automatically revise their prompt using Gemini.

In addition to prompt analysis and revision, PromptBuddy offers:

A History view of the last 10 analyzed prompts, along with associated risks and confidence scores.
A digestible Analytics Dashboard with charts showing common intents/risks and how the impact confidence scores.

How we built it

Frontend: Built with React, Tailwind CSS, and data visualizations using Chart.js
Backend: Built with FastAPI (Python), integrated with the Gemini Web API
Database: Used an SQLite database to store prompt analysis records for history and analytics
Deployment: Frontend hosted on Vercel, Backend hosted on Render

Challenges we ran into

Intent/Risk Normalization: LLM outputs can vary greatly in language, even when referring to the same intents and risks. Normalization was a challenging requirement to overcome to provide useful analytics to the user.
Time Constraints: Delivering an app with multiple features (Analysis, Revision, History, Analytics) in a short time frame was intense.