Inspiration
In this AI era every employee or an organization uses AI tools in their workflows, the only thing that separates a good employee to a average one is how they leverage these tools right now it is a black box for these organizations of how these tools are being leveraged and we want to bring observability to this exact issue.
What it does
SusCoder keeps a track of your prompts in a session and evaluates heuristically if these prompts categorize as a vibe coded session or a session where these tools are properly leveraged in the given context window. It observes the patterns over the session and profile each into one of the four personas.
How we built it
We made use of the underrated feature of AI tools which is saving the prompts into a specified location. We take this and store it in our TigerDB and have a unsupervised clustering parallelly have the agent evaluate the prompts a person used in the session and determine how effectively they have used these tools. The employer on the other side has a analytical dashboard to observe the AI tools usage in real time.
Challenges we ran into
The main challenge was to evaluate the prompts and improve its accuracy where there are very few false positives. Hallucination through limited context windows.
Accomplishments that we're proud of
Custom agent which determines heuristically which is vibe coded or not, give a 6 dimension analysis oh each prompt. Detect passwords and other personal information leaked immediately.
What we learned
The problem is a glacier, looks small on the outside but can go very deep into the waters.
What's next for SusCoder
Provide employees a plan to improve their skills
