Cleanup

What it does

cleanup turns a photo of something you want to throw away into an actual disposal plan.

Take a photo and cleanup can:

📸 Identify it → ⚠️ decide how it should be handled → 📍 find a suitable nearby destination → ✅ plan the drop-off → 📊 track the completed action

Instead of stopping at “this is plastic” or “this is a battery,” cleanup tries to answer the question that actually matters:

“What should I do with this now?”

A user can photograph one or multiple items. cleanup identifies the material, detects whether special handling is required, gives preparation instructions, and finds nearby recycling or disposal facilities whose published information matches that type of waste.

For things such as batteries, electronics, medical waste, chemicals, and other hazardous materials, deterministic safety rules can override unsafe AI suggestions.

Why cleanup is different

Most AI waste tools end with recognition.

cleanup continues into action.

The core workflow is:

SEE → UNDERSTAND → ROUTE → ACT → VERIFY → IMPACT

The AI helps understand the photo, but it does not control the entire system.

Facility matching, hazardous-waste safeguards, rate limits, proof validation, and impact calculations are handled separately and deterministically.

That makes cleanup more than an image classifier or chatbot — it is a small waste-action network connecting recognition with real-world disposal.

Inspiration

Recycling sounds simple until you actually have an object in your hand.

An old charger, battery, plastic bottle, broken electronic device, medicine container, or mixed-material product can all require completely different handling.

The information is usually fragmented:

one service tells you what the object is
another explains recycling rules
another provides a map
and the user still has to figure out whether a particular location actually accepts that material

We wanted to connect those disconnected steps into one simple workflow starting with a photo.

The goal became:

Make responsible disposal require less research than throwing something away.

How it works

1. Photograph the waste

Users can upload or take a photo directly from their phone.

cleanup can detect multiple discardable objects from the same image instead of forcing the user to scan everything separately.

2. Understand the object

A server-side Featherless multimodal AI pipeline analyzes the image and returns structured information including:

object and material
waste category
recyclability and reuse possibilities
preparation steps
special-handling requirements
qualitative certainty
facility-search tags

cleanup automatically routes requests between its internal pool of vision-capable models. The user only provides one Featherless API key when deploying and never has to choose a model.

3. Apply safety rules

AI output is not blindly trusted.

cleanup contains deterministic overrides for sensitive categories including:

batteries, e-waste, medical waste, chemicals, sharps, and hazardous materials.

For example, a detected battery cannot simply be changed into normal-bin waste because an AI response suggested it.

4. Find somewhere it can actually go

Users can use GPS or search for an area.

cleanup searches OpenStreetMap data through Nominatim and Overpass, then ranks nearby facilities according to the material the user selected.

If several locations match an address search, cleanup asks the user to choose instead of silently guessing.

5. Turn the result into an action

The user can select a compatible facility, schedule a drop-off, add the estimated weight, and save the action.

The history stays locally in the browser, making the MVP usable without requiring an account.

6. Make the result tamper-resistant

For eligible real actions, cleanup can create a signed chain:

analyzed item → matched facility → planned action → completed action

The server signs and later re-validates those receipts before an action contributes to impact totals.

This prevents someone from simply editing browser storage and creating fake impact.

Importantly, we describe this accurately:

server-attested does not mean a recycler physically verified receiving the waste.

Physical handoff remains self-reported until recycler-side QR/account verification is added.

How we built it

cleanup is a mobile-first Progressive Web App with a Node.js backend.

The frontend is intentionally lightweight and uses HTML, CSS and vanilla JavaScript.

The backend handles:

Featherless multimodal image analysis
automatic model fallback
image validation
deterministic safety normalization
recycling-facility discovery
Nominatim geocoding
Overpass queries
material compatibility ranking
API rate limiting
request cancellation and timeouts
signed action proofs
impact verification
security headers
PWA delivery

Mapping is handled using Leaflet + OpenStreetMap.

The application is designed to deploy as a single Node web service on Render, and the repository includes a complete Render Blueprint.

Challenges we ran into

AI recognition is the easy part

The difficult question was not:

“Can AI recognize a battery?”

It was:

“How do we turn that recognition into advice we can safely act on?”

That led us to separate AI understanding from deterministic disposal and safety decisions.

Real recycling data is imperfect

OpenStreetMap facilities do not always publish complete accepted-material information.

We therefore distinguish between:

published material match
unknown acceptance
no published match

instead of pretending every nearby recycling point accepts everything.

AI services fail

Models can be unavailable, cold, rate-limited, or inaccessible.

cleanup therefore automatically routes between multiple internal Featherless vision models while requiring only one API key from the deployment.

Local data should not equal trusted data

The application stores history locally for simplicity, but localStorage can be edited.

That led to the signed proof chain and server-side revalidation used by the impact dashboard.

Real browser behavior creates strange bugs

A large part of development became reliability work:

multiple tabs writing simultaneously
stale async requests
cancelled geolocation calls
slow external APIs
malformed upstream responses
repeated form submissions
corrupt localStorage
mobile overflow
PWA cache upgrades
offline behavior
large phone photos

We built regression tests around these cases rather than relying only on the happy path.

Accomplishments that we're proud of

The biggest accomplishment is that cleanup does not stop at AI recognition.

We built the complete journey from:

photo → understanding → safety → facility → planned action → completion → impact

We are especially proud of:

deterministic safety overrides instead of blindly trusting AI
automatic Featherless model routing behind one API key
material-aware real facility matching
a signed proof chain for tamper-resistant impact
privacy-conscious POST-based location lookups
offline/demo fallbacks
mobile-first PWA behavior
bounded storage and multi-tab protection
automated CI across Node.js 20, 22 and 24
an extensive regression suite built while repeatedly stress-testing edge cases

What we learned

The biggest lesson was that adding AI is much easier than building a trustworthy workflow around AI.

Recognition alone does not solve the user's problem.

A useful system also needs:

validation, uncertainty, safety rules, real-world data, failure handling, privacy, and a clear action after the prediction.

We also learned that public recycling data is inherently incomplete. A responsible product should communicate that uncertainty instead of converting incomplete data into false confidence.