Inspiration
Citizen science can expand freshwater monitoring, but citizen observations may sometimes be inconsistent, incomplete, or difficult for researchers to review at scale. We wanted to build a system that supports better-quality observations without allowing AI to silently replace human judgment.
What it does
AquaVerify AI is a human-in-the-loop quality-control system for citizen stream assessments.
A citizen submits structured observations about stream conditions, including habitat, water appearance, flow, vegetation, drainage, sewage discharge, construction pressure, and an overall ecosystem assessment.
AquaVerify then applies multiple review layers:
- basic input validation,
- transparent consistency rules,
- reference-data rarity analysis,
- Isolation Forest and One-Class SVM novelty detection,
- evidence-based explanations,
- citizen edit or override,
- environmental concern indicators,
- SQLite audit logging,
- and researcher triage with documented human review.
The system does not automatically change citizen observations and does not claim to diagnose pollution or laboratory water quality.
Track Alignment
AquaVerify AI was developed for Track 3 — AI-Supported Assessment.
Track 3 focuses on responsibly using AI to support stream assessment without replacing human judgment, particularly when citizen observations may be inconsistent or error-prone.
AquaVerify addresses this challenge through validation checks, explainable consistency analysis, reference statistics, unsupervised novelty detection, citizen review, and researcher oversight.
How it works
- A citizen submits a structured stream observation.
- Basic validation checks completeness and valid categories.
- Consistency rules identify known conflicts.
- Reference statistics identify values that are unusual within the citizen's overall assessment category.
- Isolation Forest and One-Class SVM screen for unusual multivariate observation patterns.
- AquaVerify explains the evidence to the citizen.
- The citizen can edit the observation or keep the original answers.
- Environmental concern indicators are separated from data-quality signals.
- The original submission, AI evidence, citizen decision, and final observation are stored in an SQLite audit trail.
- Researchers can review full observations, AI evidence, and citizen revision history before recording a triage decision.
Human-in-the-Loop Design
AquaVerify never silently changes citizen data.
If a consistency or novelty signal is detected, the citizen is shown the reason and can either:
- edit the observation, or
- keep the original answers.
Researchers also remain in control. They can inspect the full citizen submission, AI screening evidence, and citizen review history before marking a record as reviewed, recommending follow-up, or recording no further action.
AI / ML Approach
AquaVerify combines:
- transparent rule-based consistency checks,
- conditional rarity statistics,
- Isolation Forest,
- One-Class SVM.
The anomaly models are used to detect unusual observation patterns relative to the prototype reference dataset. They do not determine whether a citizen is right or wrong.
Data
Synthetic reference data were used to develop and stress-test the prototype workflow.
The synthetic data are not presented as scientific ground truth. Production deployment would require calibration and validation with expert-reviewed real-world stream observations.
Responsible AI
AquaVerify separates two concepts:
- data-quality review,
- environmental concern triage.
A concerning stream can still be reported accurately, so poor environmental conditions do not automatically imply poor-quality citizen data.
AquaVerify does not diagnose pollution, drinking-water safety, or laboratory water quality.
Built With
Python, Streamlit, pandas, NumPy, scikit-learn, Isolation Forest, One-Class SVM, SQLite, joblib, Pytest, and GitHub.
Challenges we ran into
One major challenge was avoiding a design where AI became the final decision-maker. We therefore separated citizen-data consistency, environmental concern indicators, and researcher triage into different layers.
We also designed the system so that subjective overall ecosystem assessments are checked separately from the multivariate ML inputs, reducing shortcut learning.
Accomplishments that we're proud of
- Built a working end-to-end human-in-the-loop prototype.
- Combined deterministic checks, statistics, and lightweight ML.
- Preserved before-and-after citizen review evidence.
- Added researcher triage and audit history.
- Deployed the application publicly with Streamlit.
- Added automated tests and a preflight validation process.
What we learned
We learned that responsible AI for citizen science is not mainly about using a larger model. It is about using the right model for the right task, explaining review signals clearly, preserving human control, and keeping scientific claims within the limits of the data.
What's next
Future work includes:
- calibration using expert-reviewed real-world OneAquaHealth observations,
- managed persistent cloud storage,
- role-based researcher access,
- optional photo-assisted observation review,
- integration with existing citizen-science and OneAquaHealth data infrastructure.
Log in or sign up for Devpost to join the conversation.