Vedaxi - Research Integrity desk

Solving truth using WEBMCP by Vedaxi

Inspiration

We started this hackathon with a much bigger idea.

Way too big, as it turned out.

We wanted to explore Elastic Web. But once we started breaking down what would actually have to exist underneath it, three days suddenly looked very small.

Before you can make the web dramatically faster or more intelligent, there is a boring but fundamental problem: information coming from different places has to remain trustworthy. If two systems disagree, an agent can't just pick one, merge them and confidently move on.

So we took that one problem and made it the hackathon.

That became Research Integrity Desk.

We thought narrowing the scope would make the build manageable.

It did not.

What it does

Research Integrity Desk is our attempt at the first layer of that larger problem: keeping information understandable and trustworthy when AI is working across different sources.

Instead of treating everything an agent finds as one giant pool of context, we care about where information came from, what changed, what disagrees and what still needs a human decision.

It is intentionally a much smaller problem than Elastic Web.

But after building it, we are fairly convinced it is one of the problems that has to be solved first.

How we built it

The funny thing is that even the "smaller" project immediately became an orchestration problem.

We built with WebMCP and used AI heavily throughout development. Codex was useful, but painfully slow for the pace of a three-day hackathon. There were limits on how many agents we could effectively keep working, so we started building infrastructure around the infrastructure.

First an orchestrator.

Then a workspace to keep track of what everybody was doing.

Then graphs and loops because agents would lose context.

Then checks because agents would confidently say something was finished when it wasn't.

Then evals because checking everything manually stopped being realistic.

Then skills.

Then enough skills that we needed a skill router.

Then memory buckets so agents got the context they needed without drowning in the entire project.

Somewhere in there we realised that managing the agents had become a genuine engineering problem of its own.

Challenges we ran into

And then we started running out of credits.

ChatGPT limits became a constraint. Codex was slow. Work moved between different tools and models depending on what could still actually run.

Anti-Gravity became part of the workflow partly because it still had credits.

Anti-Gravity was also a hallucination machine.

It could very confidently announce that something had been implemented when we would open the product and discover that, in reality, absolutely nothing had happened.

So now we needed a controller.

Hermes went above the other agents as an outside critic. I deliberately wouldn't let it build. Its job was to be obnoxious: question everything, critique everything, find the smallest inconsistencies and check whether the work agents claimed to have done actually existed.

Strangely, that worked.

Buzz helped us keep track of the moving pieces. Agent Grid helped as work spread across agents. Evals and verification loops became increasingly important because after enough hours of nonstop development, "the agent said it works" is not a testing strategy.

At one point, a part of the build that should have been straightforward consumed almost 28 hours once coding, testing, breaking and verifying were included.

And then there was the website.

AI repeatedly wanted to build us a beautiful presentation about the product rather than the actual working product. It kept drifting towards an interactive slide deck.

So even after writing the functionality, we had to keep forcing the real code back into the experience and testing that what judges would see was actually running.

We had scoped down because three days wasn't enough for the big idea.

Then we discovered just how much engineering it takes to make even the small idea reliable.

Accomplishments that we're proud of

Honestly: that it works.

We could have used the three days to make a beautiful simulation of Elastic Web.

Instead, we kept cutting scope until we had something we could actually implement, test and verify.

And somehow we shipped it while simultaneously dealing with model limits, exhausted credits, slow agents, hallucinated implementations and an orchestration stack that we hadn't planned on building.

Research Integrity Desk is a small part of the original idea.

But it is real.

That matters more to us than pretending we built the whole thing.

What we learned

We learned that there are layers underneath "AI can do this" that are very easy to ignore in a demo.

Who remembers what happened?

Who checks whether it actually happened?

Which agent should have which capability?

Which information should it see?

How do you know when two systems disagree?

What happens when a model confidently invents its own version of reality?

How do you keep several agents working without them undoing each other's work?

We ended up needing workspaces, orchestration, graphs, loops, skill routing, memory, controllers, checks and evals just to keep the development process reliable.

Which was slightly hilarious, because those were variations of the same problems we had started thinking about in the product.

The more we built, the less this looked like a problem you solve by simply adding a smarter model.

There is an entire systems layer around the model that matters.

Probably more than we appreciated when we started.

What's next for Vedaxi – Research Integrity Desk

The strange outcome of the hackathon is that we started with a huge idea, deliberately made it much smaller, and finished with an even better understanding of how huge the original problem actually is.

Research Integrity Desk is one layer.

There are many more.

Some became obvious from the product itself. Others became painfully obvious from trying to coordinate the agents building it.

Elastic Web is still the larger direction, and we are deliberately not spelling out that entire system here before we build it.

But these three days gave us something more useful than a giant prototype.

They gave us a working first layer, a much clearer map of the problems underneath it, and a fairly ridiculous list of things that have to work before the larger idea can.

We came here intending to build the big thing.

Instead, we learned what has to exist before the big thing can be built.

And somehow, between the credit limits, hallucinating agents, graphs, loops, evals and a controller whose main job was telling everybody else they were wrong, we actually shipped something.