GREM | Devpost

Architecture

Inspiration

Every retail, finance, healthcare, and legal company running search loses one in four customer queries silently. A shopper asks "shoes from the brand that sponsors Real Madrid" and the system returns sneakers from an unrelated brand. A clinician asks "drug that interacts with the medication prescribed for diabetes type 2 patients over 65" and the answer omits the critical interaction. A lawyer asks "rulings citing the precedent set by the case that established standing for environmental NGOs" and the search misses half the relevant case law. These are multi-hop queries/questions that require connecting two pieces of information. Production search systems built on retrieval systems (BM25 our baseline) fail on 27% of them. At Shopify scale that is 540 million failed queries per quarter. At a contact center that is thousands of escalations per day. At a hospital that is missed diagnoses. The industry fix is to call an LLM on every query for re-ranking. It works but costs $0.003 per query and adds 2 seconds of latency. A company processing 10 million queries per day pays $30,000 daily around $11 million a year for that fix (estimated) . And regulated industries cannot use it at all, because their data cannot leave their security perimeter. GREM is the production answer: capture the LLM's reasoning once during training using Gemini, store it as episodic memory in MongoDB Atlas, distill it into a 22 million parameter re-ranker that runs in 2-4 milliseconds with zero API calls at inference. Same recovery quality. One-thousandth the cost. Deployable on a laptop, in an air-gapped data center, anywhere data must stay local.

What it does

GREM is a quality-gated multi-hop retrieval system with three core agendas

Multi-agent verification pipeline. Three specialized Gemini agents entity overlap, bridge chain, relevance validation plus a Gemini aggregator that scores every BM25 failure on a quality scale. Only chains scoring q_final > 0.5 enter MongoDB Atlas.
MongoDB Atlas as load-bearing episodic memory. 1,145 verified reasoning chains live in Atlas with vector embeddings. Atlas Vector Search adaptively retrieves similar past chains for low-confidence queries at inference time. The same database that curates training data serves as the runtime safety net.
Distilled cross-encoder re-ranker. A 22M-parameter BERT cross-encoder fine-tuned on the verified chains, with soft labels weighted by q_final. Runs locally at 2 ms per query no LLM, no API call, no per-query cost
On 228 held-out HotpotQA bridge failures, GREM recovers 80.3% to Hits@1, with 92.5% Hits@2 and 100% Ground Rate. MongoDB Atlas Vector Search activates adaptively on 4.8% of low-confidence queries ## How we built it
Training pipeline: 1,629 dev hard failures processed through Gemini 2.5 Flash-Lite (Agents A/B/C) and Gemini 2.5 Flash (Aggregator) on Google Cloud Agent Platform
Episodic memory: 1,145 verified chains in MongoDB Atlas M0, indexed with 384-dim embeddings for Atlas Vector Search
MongoDB MCP Server: bulk insertions and verification queries route through MCP over JSON-RPC for standardized agentic memory access
Re-ranker: cross-encoder fine-tuned with custom soft-label loss
Frontend: React + Tailwind on Vercel, reads metrics and demo traces live from MongoDB Atlas

Challenges we ran into

Aggregator JSON output truncation when Gemini wrapped reasoning in markdown solved with response_format={'type': 'json_object'} and a robust two-brace JSON parser
MongoDB MCP response parsing the server wraps results in untrusted-user-data tags with confusing duplicate tag mentions in warning prose, requiring direct JSON array search rather than tag regex
Adaptive Atlas threshold tuning, tested 2.0, 0.4, 0.1; landed at 0.1 (4.8% invocation rate) which preserves baseline performance while keeping Atlas load-bearing at inference

Accomplishments that we're proud of

Complete quality-gated pipeline converting noisy BM25 failures into high-fidelity training signal
Full Google Cloud + MongoDB Atlas integration with zero per-query API cost at inference
Production deployable 22M parameters,runs on a laptop

What we learned

Episodic memory in MongoDB Atlas does not just store data it actively shapes inference quality. The same Atlas instance plays two roles: training data substrate and runtime safety net. This dual role is what makes Atlas load-bearing in the architecture, not just a database we happened to pick. We also learned that distillation can be 'too successful' the cross-encoder, once trained on Atlas chains, no longer benefits from runtime hint injection on confident queries. We turned this into a feature: adaptive Atlas activation only when the re-ranker is uncertain. The student no longer needs the teacher, except for the hard cases.