Evidentia - Your valyuble sourcing agent

Poster

Inspiration

Just like food that’s sourced well is healthier, decisions made on well sourced LLM outputs are more trustworthy and more accurate.

What it does

Our agent Evidentia is built by Sourced where we break down the prompt of an LLM and collect reliable sources through valyuble. After clustering and processing sources we then produce these sources along with the prompt to an LLM reducing token usage by upto 70%! Our sourcing journey doesn’t end here through. We need to make sure that the LLM actually uses the sources we provide and a user can guarantee fine-tuning of sources. So, we map the output to a knowledge graph and connect each piece of the output to the source where it came from so you can see, how your output is sourced!

How we built it

We use the Valyu Search API to ground the LLM output to the evidences. We formed Knwledge graphs out of the input, the evidence and the LLM outputs and calculated how strong the evidence is for the output and similarly, how correlated is the input itself with the output. We provided this as a playable application, where the user can tweak settings and recalculate their confidence. This way, the user always has confidence in the Agent's outputs!

Future scope

We believe that this knowledge graph approach can be extended not only for grounding search agents but any agents - providing the user confidence in their AI outputs We can also incorporate chain of thoughts in the graph produced to further increase the confidence in the LLM output, and go beyond agents