Echo Prompt (The Self Improving Prompt Engine)

An Auto-Prompt-Optimization project to make lives easier. This project dives into Auto Prompt Optimization to learn trends as well as apply them by refining prompts in real time to get better results.

Comment

Inspiration

I, as a high-school student am deeply interested in AI and had represented India at the International Olympiad. During a talk with one of my mentors, she had mentioned Automatic Prompt Optimization techniques and how prompts could be refined too. It was interesting to know that work was not only being done in refining the models to refine the output but the inputs were being refined too. This led to me deciding to participate and build an engine that optimizes prompts by straight out first going through hefty process of refining lots of prompts and then just learning the pattern in the prompts to lead to better results.

What it does

This project basically aims to become a middle pipeline that can be easily implemented in most LLM interfaces to become an interface which would basically convert your raw rough prompts with lots to errors into perfect prompts which lead to the best results from a prompt possible. The resultant outputs are not just better, I aimed to create prompts that lead to more factually correct, less hallucinating, more natural answers.

How I built it

I implemented a backend which consists of 3 models - GPT-oss-120b as responder, Llama-3.1-8b-instant as the objective critic for now until I can find a better alternative, GPT-oss-120b as the prompt refiner. How this works is by using the responder to respond to the prompt. Then the critic takes out errors in the output in relation to prompt and rates different aspects like factuality, natural tone, relevance to prompt, complexity and more and gives each topic a score. On the basis of these scores and feedback, the refiner refines the prompt to get a better output aiming to reach the top of the score curve. This process was done over 500+ prompts to get a database of raw prompts and refined good prompts. Then GPT-oss-20b was finetuned over these pairs of prompts to learn the pattern and automatically give the better prompt. And in the end, both the process of iterative refining and the finetuned GPT-oss-20b have been included in the website to test out. Both give good results and the backend is made in an easy an manageable way.

Challenges we ran into

I personally have never worked with connecting APIs and frontend. In the time I have been doing AI, I have always handled and worked with models and all in Google Colab like environments. But I still tried to get this working despite facing numerous challenges and several personal complications leading to delay. In the end even after lots of setback, I am here 2 hours before the deadline, making the submission.

Accomplishments that we're proud of

Despite all the challenges, I succeeded in not only making a proper backend + frontend but also created a good project that not only looks good but is great in value too. In the way, I learnt a lot about projects in general as well as a lot about frontend. What we were able to achieve with the project is something worth appreciating with a great model that can give great results just from tuning the outputs.

What we learned

Through this project I realized how different it is to move from just experimenting with models in isolation to actually building a complete product. I learned how to design a full pipeline that connects a responder, critic, and refiner together, and how important iteration is in turning even rough prompts into strong ones. On the technical side, I picked up a lot about working with APIs and connecting a backend to a frontend—something I had never done before. I also learned how to think beyond raw accuracy and instead evaluate prompts on things like factuality, natural tone, fluency, and relevance, which was both challenging and eye-opening. This project also taught me the importance of working within constraints. With limited compute, time, and setbacks along the way, I had to carefully select models and streamline my approach. Most of all, I learned how to stay resilient and adaptable, cutting unnecessary complexity and still delivering a working project I am proud of. Finally, refining over 500+ prompts gave me a much deeper understanding of how “human-like” phrasing in prompts can dramatically improve model outputs. It showed me that prompt engineering is just as much about communication and naturalness as it is about correctness.

What's next for Echo Prompt (The Self Improving Prompt Engine)

I plan to write a paper on the patterns observed in a conference and also to go forward in trying to partner this interface with different large LLMs like: Ta-da! OpenAI and all so that the general user can have a better and smoother experience.

Built With

Share this project:

Updates