ModelScope

Equations have limits. Find them.

Comment

Inspiration

Learning an equation is different from knowing when to trust it.

Students often memorize scientific models without understanding their assumptions, limitations, or valid regimes. Even a high fit score can conceal systematic disagreement between a model and actual measurements.

ModelScope addresses this problem by making model adequacy an evidence-based learning task: When should we stop trusting this approximation for these measurements?

Rather than treating equations as universally reliable or relying on AI-generated answers, ModelScope connects scientific concepts to observations and student-owned experiments. It responds to the AI + Education challenge by helping students investigate not just how models work, but where their limitations become visible.

What it does

ModelScope is an interactive scientific learning platform that helps students evaluate whether mathematical models adequately explain experimental measurements.

It compares observations against a configured scientific baseline, searches continuous segmented alternatives, and presents the supporting numerical and visual evidence.

Students can:

  • Explore four interactive 3D scientific systems and investigate how scientific relationships behave.
  • Analyze their own measurements through CSV uploads, pasted tables, or manual data entry.
  • Inspect linked plots, residuals, and measurements to identify patterns that a simple fit score might conceal.
  • Compare scientific baselines against segmented alternatives to investigate possible changes in model behavior.
  • Examine evidence-based outcomes that distinguish supported transitions, ambiguous results, no clear transition, and insufficient evidence.
  • Export reproducible JSON reports containing provenance and numerical evidence.
  • Use optional Gemini assistance to configure supported analyses and understand existing findings without allowing AI to determine scientific results.

The goal is not to declare that a model is universally correct or incorrect. It is to help students understand what the available measurements actually support.

How we built it

ModelScope uses a deterministic scientific analysis engine written in pure TypeScript.

The engine performs:

  • Baseline fitting or fixed-theory prediction.
  • Residual analysis to identify systematic disagreement.
  • Penalized comparisons with continuous segmented alternatives.
  • Sustained same-direction disagreement checks.
  • An influential-observation safeguard.
  • Transition sensitivity analysis.
  • Reproducible JSON reporting with preserved numerical evidence and provenance.

The same complete set of observations remains in the reported fits, avoiding selective exclusion of inconvenient measurements.

The reported transition sensitivity range reflects sampling and candidate sensitivity; it is not a statistical confidence interval.

Gemini AI Integration

Gemini provides an intentionally restricted AI layer.

Given a student's experiment description and exact column headers, Gemini can propose a supported analysis configuration. However, the student must review and confirm that configuration, provide a justified response scale, and separately initiate the deterministic analysis.

Gemini can also explain an existing finding using limited summary context.

Critically, Gemini cannot:

  • Fit the scientific model.
  • Select a breakpoint or transition.
  • Invent uncertainty estimates.
  • Modify numerical evidence.
  • Change whether a finding is supported.

AI explains. The analysis engine decides.

This separation keeps scientific conclusions reproducible, inspectable, and independent of generative AI.

Challenges we ran into

One of the largest challenges was distinguishing sustained model disagreement from an attractive numerical fit improvement or the influence of a single observation.

A more flexible model can achieve a better fit without providing convincing evidence of a meaningful transition. We needed safeguards that encouraged students to examine residual structure and evidence rather than accept improved fit scores uncritically.

Other challenges included:

  • Keeping AI-generated configuration proposals understandable and reviewable.
  • Ensuring AI explanations remained faithful to the current deterministic finding.
  • Preserving reproducibility throughout the analysis and export workflow.
  • Presenting dense numerical evidence without overwhelming students.
  • Making four interactive 3D scientific systems and linked evidence views usable on desktop and mobile.

The central challenge was balancing scientific rigor with an interface accessible to learners.

Accomplishments that we're proud of

ModelScope is a working public educational application, not merely a concept or interface mockup.

Its completed features include:

  • Four interactive 3D scientific systems for exploring scientific relationships.
  • Custom-data analysis through multiple input methods.
  • Linked visualizations and numerical evidence for investigating model adequacy.
  • Deterministic model comparison and residual analysis with safeguards against misleading conclusions.
  • Reproducible JSON exports preserving analytical evidence.
  • Bounded Gemini integration with explicit student confirmation.
  • Responsive desktop and mobile interfaces for exploring scientific evidence.

Verified Technical Results

The verified release has 306 passing tests, with TypeScript typechecking, linting, and the production build passing.

The demonstration uses recordings of the actual production application, including the confirmation workflow required before an AI-proposed configuration can be used.

Most importantly, ModelScope makes a frequently overlooked scientific question approachable for students:

Does this equation actually remain trustworthy when confronted with measurements?

What we learned

Building ModelScope reinforced that scientific model adequacy depends on assumptions, sampling, and residual structure.

A better fit alone does not establish a transition, and a detected transition does not establish a physical cause.

Scientific conclusions require more than favorable numerical scores.

The project also demonstrated the value of separating generative AI from deterministic scientific computation.

AI can help students understand and navigate technical evidence without becoming the authority that produces the scientific conclusion.

By keeping calculations reproducible and explanations constrained to existing results, ModelScope encourages students to question evidence rather than accept answers automatically.

What's next

The next goal is to evaluate ModelScope's educational experience with teachers and students.

Future work includes:

  • Classroom dataset testing: Analyze independently collected experimental measurements.
  • Educational evaluation: Study how students interpret scientific assumptions and model limitations.
  • Statistical evaluation: Examine the suitability of the current analytical assumptions across different experimental settings.
  • Interface improvements: Refine the experience based on feedback from teachers and students.
  • Expanded scientific coverage: Investigate additional scientific systems and supported analytical configurations.

These are proposed future directions, not completed classroom validation studies.

Responsible AI and Privacy

Gemini proposes supported configurations that require explicit student confirmation and explains existing deterministic evidence.

It cannot select transitions, fit models, invent uncertainty, or change analytical support outcomes.

Raw measurements remain in the browser during normal analysis. Optional AI requests send an experiment description and exact column headers, or a selected finding with limited summary context.

Privacy disclosure: Free-tier AI inputs may be used to improve Google products, so users should avoid including sensitive information in optional AI requests.

Limitations and Disclosure

ModelScope is an educational research prototype, not a general-purpose scientific validation system.

Its built-in datasets are synthetic educational demonstrations, not laboratory validation.

The current implementation supports restricted baseline models, one possible hinge, ordinary least squares, and assumed response scales.

It does not provide:

  • Calibrated false-positive rates.
  • Statistical confidence intervals for transitions.
  • Inferred physical mechanisms.
  • Completed classroom outcome studies.

These limitations are important because the application is intended to teach students how to interpret scientific evidence responsibly, not to imply greater certainty than the underlying methods justify.

Development Disclosure

ModelScope was developed independently by Luis Contreras as a solo participant, starting from scratch during the overlapping eligible hackathon period.

Retained Git history begins October 4, 2026.

Built With

Share this project:

Updates

Submission history