SnapTrash — Project Story

Inspiration

It started in the hackathon room itself. Every table had food scraps, plastic cups, and wrappers — all dumped into the same bin. Nobody was separating anything, and that wasn't laziness. Restaurants are one of the fastest-paced environments on the planet. Even when owners want to segregate waste, there is simply no time — orders are flying, staff are stretched, and the bin gets whatever lands in it first.

That moment made us ask: what if the bin could understand itself? What if instead of relying on humans to classify waste at the point of disposal, we let an AI reason through it — continuously, automatically, and at scale?

The numbers made the problem impossible to ignore. The US generates over 80 billion pounds of food waste annually, and restaurant plastic waste contributes meaningfully to the $300B+ annual cost of landfill overflow. The root cause in both cases is the same: no visibility, no data, no feedback loop. We built SnapTrash to close that gap.

What We Built

SnapTrash is an end-to-end AI waste intelligence platform. A camera mounted on a restaurant bin captures an image every 20 seconds. That image travels through a cloud-native, event-driven pipeline that classifies every food item and plastic artifact in the bin, computes sustainability metrics, forecasts future waste trends, and automatically triggers real-world remediation when thresholds are exceeded.

How We Built It — Technical Deep Dive

1. Ingestion Layer — Event-Driven Image Pipeline on AWS

Every image upload lands in an AWS S3 bucket (snaptrash-raw-incoming). An AWS Lambda function fires on every s3:ObjectCreated event. This serverless trigger is the heartbeat of the entire system — it reacts to reality rather than polling for it.

Deduplication via Amazon Rekognition

The first challenge in any continuous imaging system is deduplication. A bin photographed every 20 seconds produces enormous redundancy — most consecutive frames are near-identical. Sending every frame downstream for inference would be computationally wasteful and economically untenable.

We solve this with a perceptual deduplication layer using Amazon Rekognition's DetectLabels API. On each Lambda invocation:

The new image is compared against the last analyzed frame, whose metadata is stored in Amazon DynamoDB for $O(1)$ lookup by bin_id.
Rekognition returns a similarity confidence score $S \in [0, 1]$.
We apply a deduplication threshold $\tau = 0.85$:

$$ \text{process}(I_t) = \begin{cases} \text{true} & \text{if } S(I_t, I_{t-1}) < \tau \ \text{false} & \text{otherwise} \end{cases} $$

Images that pass the threshold are promoted to snaptrash-analyzed and queued for inference. This eliminates approximately 80% of redundant frames before a single AI call is made.

Dynamic Batching for Inference

Rather than invoking the reasoning model per image in isolation, we implement dynamic batching across concurrent bin uploads. Incoming images that arrive within a configurable window $\Delta t = 5s$ are grouped into a single batch request to the xAI Grok 4.20 reasoning model:

$$ \text{batch}(t) = { I_i \mid t - \Delta t \leq t_i \leq t } $$

This reduces API overhead, amortizes prompt tokens across multiple images, and significantly lowers per-image inference latency under high concurrency. The batch size is bounded at $B_{\max} = 10$ images to stay within context window constraints.

2. Reasoning Layer — xAI Grok 4.20

Unlike a standard vision classifier, Grok 4.20 is a frontier reasoning model. It doesn't just label objects — it reasons through the image in a structured chain of thought, returning a fully typed JSON response per bin image containing:

Food items: type, decay stage (fresh / wilting / decayed), estimated weight $\hat{w}$ in kg, dollar value lost, CO₂ impact in kg, and remaining shelf life in days
Plastic items: resin code (PET, HDPE, PVC, LDPE, PP, PS, Other), polymer classification, color, black plastic flag, recyclability, and EPA harmful/banned status

The shelf life computation leverages USDA decay models. For a food item $f$ with observed decay stage $d \in {0, 1, 2}$, the estimated remaining shelf life $L$ is:

$$ L(f, d) = L_{\max}(f) \cdot \left(1 - \frac{d}{2}\right) \cdot e^{-\lambda \cdot T} $$

where $L_{\max}(f)$ is the USDA baseline shelf life, $\lambda$ is a decay constant, and $T$ is ambient temperature. This tells restaurant owners whether organic waste can safely sit for another 2–3 days before composting — reducing unnecessary bin trips.

3. Storage and Caching — Databricks Delta Lake

All structured inference output is written to Databricks Delta Lake via the SQL Warehouse API. Delta Lake serves as the single source of truth for the entire platform — every scan, aggregation, forecast, and alert is stored here with full ACID transaction guarantees.

Write-Path Caching

To avoid hammering the SQL Warehouse on every individual scan write, we implement a write-side cache in the FastAPI backend using an in-memory buffer. Scan records are accumulated in a local queue and flushed to Delta Lake in micro-batches every $N = 50$ records or every $T_{\text{flush}} = 10s$, whichever comes first:

$$ \text{flush if } |\text{queue}| \geq N \text{ OR } (t_{\text{now}} - t_{\text{last flush}}) \geq T_{\text{flush}} $$

This reduces SQL Warehouse connection overhead by roughly 10× under sustained load while keeping data freshness within a 10-second window — acceptable for a waste analytics use case.

Read-Path Caching for Dashboards

Dashboard queries — sustainability scores, weekly aggregations, locality maps — are expensive if recomputed on every frontend request. We cache these at the API layer with a TTL-based invalidation strategy:

Locality aggregations: TTL = 5 minutes
Sustainability scores: TTL = 10 minutes
Forecast outputs: TTL = 1 hour (Prophet model runs on a scheduled job)

This keeps the React frontend snappy without over-querying Databricks during peak usage.

4. Scheduling — Databricks Workflows

Three automated jobs run on Databricks Workflows:

Locality Aggregation Job (runs every 30 minutes)
Computes rolling 7-day plastic and food waste totals by zip code across all restaurants. Aggregates into the locality_agg Delta table.

Threshold Check Job (runs every hour)
Evaluates locality plastic levels against threshold $P_{\text{threshold}} = 150\text{ kg/week}$ and per-restaurant threshold $P_{\text{restaurant}} = 30\text{ kg/week}$:

$$ \text{alert}(z) = \begin{cases} \text{true} & \text{if } \sum_{r \in z} P_r^{(7d)} > P_{\text{threshold}} \ \text{false} & \text{otherwise} \end{cases} $$

When triggered, the system identifies the nearest plastic-eating enzyme lab from a geo-indexed CSV, fires an SMTP alert email, and initiates a Vapi voice call — all with no human in the loop.

Prophet Forecasting Job (runs daily)
A Facebook Prophet time series model is trained on five years of US Municipal Solid Waste data from the Dryad dataset, tracked and versioned with MLflow on Databricks. It generates 7-day ahead forecasts of food and plastic waste per restaurant:

$$ \hat{y}(t) = g(t) + s(t) + h(t) + \epsilon_t $$

where $g(t)$ is the trend component, $s(t)$ captures weekly seasonality (restaurants waste more pre-weekend), $h(t)$ accounts for holiday effects, and $\epsilon_t$ is the error term.

5. Frontend

The dashboard is built in React + TypeScript + Vite, styled with Tailwind CSS, charted with Recharts, and mapped with Mapbox GL. Locality sustainability scores are rendered as a choropleth layer over San Diego neighborhoods — giving consumers a geographic view of restaurant environmental performance alongside Yelp ratings.

Challenges

Deduplication sensitivity tuning was harder than expected. A threshold $\tau$ that's too high misses real bin changes. Too low and we over-trigger inference on noise — a slight lighting shift, a shadow, a bag shifting. We ran empirical tests across 50 staged bin images to land on $\tau = 0.85$ as the sweet spot.

Batch inference timing introduced latency tradeoffs. Waiting for a full batch window of $\Delta t = 5s$ improves throughput but hurts responsiveness for single-bin scans. We solved this with an adaptive batching policy — if a bin hasn't received a companion image within $2s$, it's dispatched as a singleton rather than waiting for the full window.

Delta Lake write contention under simulated load from 100 concurrent restaurants caused occasional transaction conflicts. We resolved this by partitioning the scans table by (date, locality_zip) and enabling Delta Lake's optimistic concurrency control, which reduced conflict rates to under 0.3%.

Grok structured output consistency required careful prompt engineering. Reasoning models are verbose by default — we had to constrain output format strictly via system prompt and implement a JSON validation + retry loop on malformed responses, with a fallback to partial extraction when only some fields were parseable.

What We Learned

Building SnapTrash taught us that the hardest part of a real-time AI system isn't the model — it's everything around it. Deduplication, batching, caching, scheduling, and write contention are the problems that separate a demo from a production system. We learned to think in data flow, not features — every architectural decision had to answer: what happens at 10,000 bins? And we learned that a frontier reasoning model like Grok 4.20 changes what's possible at the analysis layer — multi-attribute structured reasoning over an image in a single call is something classifiers simply cannot do.