ClimateReview AI — Agentic Scientific Review

An evidence-grounded Gemini agent for scientific reviewers: navigate complex climate documents, analyze passages, trace context, and draft human-reviewed comments without automatic submission.

Comment

ClimateReview AI — Agentic Scientific Review

Inspiration

Scientific review is one of the most demanding knowledge-work processes.

A reviewer may need to work through hundreds or thousands of pages, understand a statement in its surrounding context, locate related evidence, assess whether uncertainty is expressed correctly, identify inconsistencies, and finally convert that analysis into a concise and useful review comment.

The difficult part is not simply finding text.

The difficult part is maintaining context and evidence while moving repeatedly between:

Document → Passage → Evidence → Analysis → Comment → Human Decision

ClimateReview AI was created to explore whether an agentic system can reduce that repetitive work without removing the scientific reviewer from the decision-making process.

Our goal is not to make AI the reviewer.

Our goal is to give the reviewer an evidence-grounded collaborative agent.

What it does

ClimateReview AI provides a scientific-review workspace where a human reviewer can:

  1. Open and navigate a large scientific document.
  2. Move directly to a relevant page.
  3. Select a specific passage for review.
  4. Ask the agent to investigate that passage.
  5. Retrieve bounded supporting context from the indexed document.
  6. Analyze the passage for scientific-review issues.
  7. Generate a structured draft review suggestion.
  8. Inspect the evidence and confidence behind the suggestion.
  9. Accept, modify, reject, or retain the suggestion as a draft.

Instead of returning an unrestricted chatbot answer, the agent produces structured review output containing:

  • Issue identified
  • Supporting evidence
  • Proposed reviewer comment
  • Suggested resolution
  • Confidence level

The reviewer always remains in control.

ClimateReview AI does not automatically submit AI-generated comments to an external scientific review portal.

Why this is agentic

ClimateReview AI is not designed as a PDF chatbot.

The agent coordinates a multi-step review workflow.

Reviewer
   ↓
PDF / Scientific Document
   ↓
Selected Page + Passage
   ↓
Context Retrieval Agent
   ↓
Evidence Grounding
   ↓
Scientific Review Analysis
   ↓
Structured Comment Generation
   ↓
Confidence + Traceability
   ↓
Human Review
   ↓
Accept / Edit / Reject / Draft Queue

The system performs the context gathering and analysis that would otherwise require repeated manual searching.

This makes the agent a Collaborative Partner: it performs the heavy analytical work while the expert retains authority over the final scientific judgment.

How we built it

ClimateReview AI uses a layered architecture so that document handling, retrieval, agent reasoning, model interaction, and the reviewer interface remain separated.

Reviewer interface

The frontend is built with:

  • Next.js
  • React
  • TypeScript
  • PDF.js / react-pdf

The reviewer workspace is designed around a split-screen experience:

Left: scientific document and passage selection Right: agent analysis, evidence, confidence, and draft review comments

This is deliberately different from a traditional chat interface.

The document itself is the primary workspace.

Backend

The backend uses:

  • Python
  • FastAPI
  • Google Cloud
  • BigQuery

Scientific documents are broken into indexed chunks with document, page, and chunk metadata.

When a reviewer selects text on a particular page, the backend retrieves only bounded context associated with that document and page rather than blindly placing the complete document into the model prompt.

Agent and Gemini layer

The hackathon implementation uses Google's agentic AI stack with:

  • Gemini 3.5 Flash
  • Google Agent Development Kit / Google GenAI SDK
  • Google Cloud infrastructure

The agent receives the selected passage together with retrieved evidence and a scientific-review instruction set.

The review prompt explicitly tells the model to:

  • assess only the available evidence,
  • avoid inventing citations or supporting facts,
  • distinguish evidence from inference,
  • consider accuracy and completeness,
  • evaluate balance,
  • examine uncertainty,
  • preserve policy neutrality,
  • and return structured output.

This produces a workflow that is easier for a human reviewer to inspect than unrestricted generated prose.

Google Cloud

The application is designed to run on Google Cloud, including:

  • Cloud Run for application services
  • BigQuery for document chunks, metadata, retrieval, and analytical storage
  • Gemini for agent reasoning

The deployment architecture keeps the user interface, application services, data layer, and model layer independently manageable.

Evidence grounding

One of the most important design decisions was to avoid asking the model to review an isolated quotation without context.

For every review request, ClimateReview AI can retrieve page-level document context.

Conceptually:

Selected passage
        +
Current page
        +
Relevant document chunks
        +
Review criteria
        ↓
Gemini review agent
        ↓
Structured evidence-grounded suggestion

This improves traceability and makes unsupported suggestions easier to detect.

If the available context is insufficient to justify a substantive comment, the agent is instructed to state that explicitly rather than manufacture evidence.

Human-in-the-loop by design

Generative AI can assist scientific review, but scientific responsibility should remain with the expert.

ClimateReview AI therefore separates:

AI suggestion

from

Reviewer decision

The agent can analyze, retrieve, compare, structure, and propose.

The reviewer determines whether the suggestion is scientifically valid.

Every generated comment is treated as a draft.

This human-in-the-loop architecture is particularly important for high-impact scientific assessment workflows.

Challenges we faced

1. Large-document context

Scientific reports can exceed practical prompt sizes.

Passing an entire report into every model request is inefficient and makes grounding harder.

We addressed this by indexing documents into page-aware chunks and retrieving bounded context for each review operation.

2. Preventing hallucinated evidence

A fluent comment is not necessarily a correct comment.

We therefore separated evidence retrieval from comment generation and explicitly instruct the agent not to invent evidence when the supplied context cannot support a claim.

3. Making the interface reviewer-first

Our first instinct was similar to many AI applications: provide a chat interface.

But scientific review is document-centric rather than chat-centric.

We redesigned the workflow around:

read → select → investigate → evaluate → draft

rather than:

ask → answer

4. Separating AI assistance from official submission

Scientific review can involve controlled or confidential material.

The public hackathon demonstration therefore uses public, appropriately licensed, or synthetic demonstration content.

The prototype does not automatically submit generated comments to an official external review system.

5. Moving from an AI call to an agent workflow

Calling Gemini from an API is relatively easy.

Designing a maintainable agentic workflow required us to think about:

  • context boundaries,
  • agent responsibilities,
  • state,
  • evidence provenance,
  • structured responses,
  • human intervention,
  • failure handling,
  • and deployment architecture.

That architectural work became one of the most important learnings from the project.

What we learned

The biggest lesson was that the value of an AI agent is not simply the quality of the generated text.

For knowledge-intensive work, the real value comes from coordinating the complete workflow around the model.

For ClimateReview AI that means:

finding the right context + preserving provenance + applying review criteria + structuring the result + returning control to the expert

We also learned that restricting context can sometimes be more valuable than providing more context.

A bounded, traceable evidence window gives the reviewer a clearer basis for evaluating an AI recommendation.

Finally, we learned that human-in-the-loop architecture does not make an agent less capable.

For scientific work, it can make the system considerably more useful and trustworthy.

What's next

ClimateReview AI can evolve into a broader scientific assessment intelligence platform.

Planned capabilities include:

  • cross-chapter consistency analysis,
  • duplicate-comment detection,
  • citation and reference verification,
  • figure and table analysis,
  • multimodal scientific review,
  • comparison of statements across report versions,
  • reviewer-specific working memory,
  • uncertainty-consistency checking,
  • evidence graphs,
  • multi-agent specialist review,
  • asynchronous document review,
  • review coverage dashboards,
  • and collaborative expert workflows.

A future multi-agent architecture could assign specialized roles such as:

Scientific Reviewer Agent
        ↓
 ┌──────────────┬───────────────┬────────────────┐
 │              │               │                │
Evidence     Citation       Consistency      Uncertainty
Agent        Agent          Agent            Agent
 │              │               │                │
 └──────────────┴───────────────┴────────────────┘
                        ↓
                Review Synthesizer
                        ↓
                  Human Reviewer

Vision

Scientific experts should spend their time exercising judgment, not repeatedly searching through documents for context they have already seen.

ClimateReview AI explores a future where AI agents perform that investigative overhead while scientists remain responsible for the conclusions.

AI does the searching, structuring, and first-pass analysis.

The scientist makes the judgment.

That is the collaborative model we built ClimateReview AI to demonstrate.

Built With

Share this project:

Updates

posted an update —

I worked across the frontend, backend, and agent architecture, but what I’m most proud of is bringing those pieces together into a reviewer-first AI workflow.

On the frontend, I built the Next.js/React workspace around a split-screen experience: the scientific document on one side and the agent’s evidence, analysis, confidence, and proposed review comment on the other. This was an important design decision because scientific review is fundamentally document-centric, not just a chatbot conversation.

The trickiest part was making the AI useful without simply sending an entire scientific report to Gemini. I worked on the page-aware retrieval approach, where a reviewer selects a passage and the system retrieves bounded, relevant context from the indexed document before asking the agent to analyze it. This helped address both large-document context limits and the risk of unsupported or hallucinated evidence.

What I’m most proud of is the shift from thinking about this as an “AI call” to thinking about it as an agentic workflow: select → retrieve evidence → analyze → structure → review. The final decision always remains with the human reviewer; AI-generated comments are drafts rather than automatically submitted decisions.

The biggest thing I learned is that building a good AI agent is not just about generating better text. The real engineering challenge is controlling context, preserving evidence provenance, defining clear agent responsibilities, handling failure cases, and designing the human-in-the-loop experience around the model.

This project reinforced something important for me: AI should do the searching, structuring, and first-pass analysis so that experts can spend more of their time exercising judgment.

Log in or sign up for Devpost to join the conversation.

Submission history