HalluciFix Agent

HalluciFix Agent detects unsupported claims in RAG answers, traces every run with Arize Phoenix, and suggests safer, evidence-backed fixes.

Comment

Inspiration

HalluciFix Agent was inspired by a problem I often run into when using AI to read and summarize complex documents: the answer can sound confident, polished, and useful, but it is not always clear whether the answer is actually faithful to the original source.

This becomes especially risky in retrieval-heavy workflows such as literature review, policy analysis, technical documentation, compliance review, and hackathon rule interpretation. A RAG system may retrieve relevant passages, but the final answer can still overstate, simplify incorrectly, or introduce claims that are not truly supported by the text. In other words, the model may sound right while subtly changing the meaning of the original document.

I wanted to build an agent that does more than generate an answer. HalluciFix Agent checks whether the answer is grounded in the retrieved evidence. It helps users see which parts of the answer are supported, which parts are unsupported, and how to rewrite the answer more safely.

What it does

HalluciFix Agent is a RAG hallucination detection agent. A user asks a question, and the system retrieves relevant source evidence, generates an answer, and then evaluates the answer at the claim level.

For each run, the agent shows:

  • the generated RAG answer
  • retrieved evidence from the source document
  • a hallucination risk score: Low, Medium, or High
  • supported claims
  • unsupported claims
  • a safer answer
  • fix suggestions
  • trace-ready metadata for Arize Phoenix observability

In the demo, I used the Google Cloud Rapid Agent Hackathon official rules as the knowledge base. For example, when a user asks whether the demo video can be longer than 3 minutes, HalluciFix retrieves the relevant rule, generates an answer, and checks whether each claim is supported by the retrieved evidence.

How we built it

I first created a RAG data source using Google Cloud AI Applications. The source document was uploaded through Cloud Storage and connected to a HalluciFix search application so the system could retrieve grounded evidence from the hackathon rules.

Then I built a lightweight web application for the hallucination detection workflow. The app is written in Python and includes document chunking, retrieval, answer generation, claim-level support checking, hallucination risk scoring, safer-answer generation, and trace logging.

The app is deployed on Google Cloud Run and connected to a public GitHub repository. I also configured Arize Phoenix environment variables so each run can be traced with the question, retrieved evidence, answer, risk score, and evaluation metadata.

Challenges we ran into

One challenge was turning “hallucination detection” from an abstract idea into something users can understand immediately. I did not want the project to be only a backend evaluation script or a simple chatbot. I wanted the interface to clearly show the relationship between the answer and the evidence.

Another challenge was balancing speed and reliability. RAG evaluation can become complex very quickly, especially when checking long answers against multiple retrieved passages. For this prototype, I focused on a clear claim-level workflow that is easy to demonstrate: retrieve evidence, generate an answer, split the answer into claims, score support, and produce a safer version.

I also had to connect several moving parts: Google Cloud AI Applications, Cloud Storage, Cloud Run, GitHub deployment, and Arize Phoenix-style observability. Getting the full path from source document to deployed web app working was an important part of the project.

Accomplishments that we're proud of

I am proud that HalluciFix Agent is not just another RAG chatbot. It adds a quality and safety layer on top of RAG answers.

The project successfully demonstrates:

  • a working Google Cloud RAG data source
  • a deployed Cloud Run web app
  • evidence retrieval from source documents
  • claim-level hallucination risk checking
  • supported and unsupported claim display
  • safer answer generation
  • trace-ready metadata for Arize Phoenix
  • an open source GitHub repository with an MIT License

The most important accomplishment is that the user can see why an answer should or should not be trusted.

What we learned

I learned that RAG quality is not only about retrieving relevant documents. Even when the retrieved context is relevant, the generated answer can still distort the meaning of the source, omit important conditions, or add unsupported details.

I also learned that AI observability is essential for real-world agent systems. It is not enough to know the final answer. Developers need to inspect the full path: user question, retrieved evidence, generated answer, evaluation result, and risk signals.

This project helped me understand how Google Cloud services and observability tools such as Arize Phoenix can work together to make AI systems more transparent and trustworthy.

What's next for HalluciFix Agent

Next, I would like to expand HalluciFix Agent into a more complete RAG monitoring and repair platform.

Future improvements include:

  • using Gemini for more advanced claim extraction and judge-style evaluation
  • integrating deeper Arize Phoenix traces and dashboards
  • supporting more document types and larger knowledge bases
  • adding citation-level comparison between claims and evidence
  • generating automatic remediation tasks when hallucination risk is high
  • supporting teams that build customer support bots, research assistants, legal assistants, compliance tools, and internal knowledge-base agents

The long-term goal is to help teams move from “AI answered my question” to “AI answered my question, and I can verify why it is trustworthy.”

Built With

Share this project:

Updates