-
-
Scan the QR code at designated areas to upload without sharing location details or signing in.
-
Take a photo and upload to the website with a single click.
-
Ambiguous or incorrect labelling can deprive downstream systems of crucial contextual data.
-
Train an intermediate classification model in real-time to provide more context and improve downstream processes.
Inspiration
When environmental emergencies or stream hazards occur, crowdsourced reporting is often broken. Users are usually forced to download an app, create an account, approve invasive GPS permissions, or navigate confusing forms. Worse, when people do upload photos, reviewers and downstream systems often receive unorganized, out-of-context images without knowing the exact vantage point. Human responders spend crucial time deciphering photos, and automated systems get tripped up by noisy, unstructured data.
We built Rill to strip away every ounce of friction from civic environmental reporting. By placing location-anchored QR codes at monitored river sites, anyone walking by can instantly snap a photo and report a hazard in seconds, no app download or account wall required. The QR code serves as an unambiguous physical location anchor, which feeds straight into an intermediate advanced classification model that turns raw photos into clean, structured telemetry for downstream responders.
What it does
Rill is a real-time stream intelligence platform that connects zero-friction field reporting with deep vision classification and active learning:
Location-Anchored Intake: QR codes posted at monitored river segments encode precise location keys. Scanning a code instantly opens the intake view locked to that specific site.
Intermediate Vision Intelligence: The moment an image is uploaded, an intermediate classification model analyzes the photo in real time, classifying it across 6 core water conditions (Clear, Murky, Flooded, Whitewater, Algae, Debris) with percentage confidence scores.
Visual Similarity Matcher: Incoming photos are mapped into an embedding space to calculate visual similarity against other river segments across the watershed.
Active Learning & Retraining in the Background: If an initial prediction is incorrect, users or reviewers can tag the true condition directly in the UI. Rill triggers an automated background task that re-trains the model on the tagged photo, adapts class centroids, updates model weights (safetensors), and re-scores existing reports on the fly without server downtime.
Stream Intelligence Dashboard: An obsidian dark-mode dashboard provides real-time site metrics, segment health badges, 7-day imagery condition trend charts, and chronological report timelines.
How we built it
Frontend: Built with React, TypeScript, and Vite, featuring a glassmorphic obsidian UI system with Plus Jakarta Sans and JetBrains Mono typography, glowing risk tier badges, and responsive trend charts.
Backend: Powered by FastAPI (Python) with Motor for async operations and MongoDB GridFS for streaming raw image storage directly inside the database.
Vision Model Architecture: Built in PyTorch using a ResNet-18 backbone paired with a linear projection layer that maps stream imagery into a 128-dimensional L2-normalized embedding space.
Metric Learning (Triplet Loss): We trained the network using triplet loss (Anchor, Positive, Negative) so that images sharing visual water conditions cluster tightly together in vector space.
Centroid Classification & Active Learning: Instead of a static classification head, condition scoring uses nearest-centroid matching against dynamically adapted class centroids. When users tag a corrected image, ml/retrain.py computes new embeddings and blends the new ground truth into the centroids in background threads.
Challenges we ran into
Structuring Metric Learning for Stream Conditions: Traditional classification heads struggle when real-world water conditions overlap (e.g., murky water that also has debris). Moving to a 128-dim triplet embedding space allowed us to output continuous similarity scores across all classes rather than a rigid binary label.
Zero-Downtime Active Learning: Wiring up background retraining (BackgroundTasks) in FastAPI so that centroid updates, safetensor weight persistence, and report re-scoring happened asynchronously without blocking live HTTP traffic or crashing the dev server.
Zero-Friction Location Verification: Ensuring the system could enforce strict, non-negotiable location validation (segmentCode) without ever demanding intrusive browser GPS permissions from the user.
Accomplishments that we're proud of
Seamless End-to-End Flow: Going from scanning a QR code on a phone to streaming the image into MongoDB GridFS, running PyTorch embedding inference in under 50ms, and updating the live dashboard in real time.
Self-Improving AI Loop: Implementing a working Active Learning pipeline where reviewer feedback directly improves model accuracy in the background.
Structured Context for Downstream AI: Creating a system that transforms messy, raw crowdsourced photos into rich, structured telemetry; giving downstream LLM agents (like emergency severity routing models) clean, pre-filtered context.
What we learned
Intermediate Models are Game Changers: Inserting a fast, lightweight vision embedding model before downstream LLM/responder workflows drastically reduces noise and prevents bad data from corrupting downstream decision-making.
Friction stalls Participation: The best civic tool is the one people can use in 5 seconds without creating accounts or approving location popups.
What's next for Rill
Rill Alert: Automated SMS and WebPush alerts triggered when segment risk tiers escalate to Caution or Emergency.
Rill Console: A specialized multi-monitor command view for municipal water authorities to dispatch field teams directly from verified report clusters.
Edge Deployment: Running lightweight ONNX versions of the StreamEmbeddingNet directly on field cameras and mobile devices for offline hazard detection.
Track Alignment: Track 6: Resilience Informatics
Rill aligns directly with Resilience Informatics by bridging community-driven climate monitoring with adaptive data science to protect local watersheds and municipal infrastructure.
Here is why Rill fits this track:
Building Community Climate Resilience: Waterways are the frontline indicators of climate emergencies. By enabling zero-friction, instant reporting without app downloads or login walls, Rill turns everyday citizens into active sensors, giving communities and responders early-warning telemetry before environmental hazards escalate into costly disasters.
Solving the "Data Chaos" Problem in Disaster Informatics: During crisis events, incoming crowdsourced data is notoriously chaotic, unorganized, and lacking context. Rill applies informatics principles to structure this data at the point of capture: physical QR codes act as trusted spatial anchors, while our intermediate vision embedding model transforms raw, unstructured photos into clean, categorized telemetry (similarity scores, condition trends, and risk tiers).
Resilient AI through Active Learning: Environmental computer vision models often suffer from domain drift when weather conditions or seasons shift. Rill features a self-correcting active learning loop that adapts class centroids in the background whenever reviewers tag corrected conditions. The AI system itself is resilient; getting smarter and adjusting to real-world edge cases without requiring complete system re-deployments.
Log in or sign up for Devpost to join the conversation.