Smart Bin Recognition

Point your phone at a bin. Learn what it is and what goes in it – in your language, anywhere.

Comment

Inspiration

Smart Bin Recognition began with a simple problem: waste-sorting rules change from place to place, but people need the answer while standing beside a bin—not later, after searching through a municipal website, PDF, sign, or unfamiliar local-language app.

The idea grew from the Deggendorf Waste Sorting Assistant, a computer-vision coursework project I built at Deggendorf Institute of Technology with Sameer and Fares. That prototype used 466 locally collected photographs, YOLOv8, and Streamlit. It proved that a camera could recognize familiar bins, but it was German-only, tied visual classes directly to disposal meanings, and could not provide a responsive live-camera experience.

Its final presentation proposed multilingual support and mobile integration. Smart Bin Recognition is my attempt to build that idea properly.

Pre-event baseline: the earlier coursework prototype. During Hacksocial 2026, I rebuilt the concept as a documented, multilingual PWA, an inference service, a reproducible ML pipeline,e and a safety-gated deployment architecture.

What it does

Point your phone at a waste bin. Smart Bin Recognition is designed to:

  • detect multiple bins in the camera frame;
  • identify each bin by its physical form;
  • measure visible colour without assuming what that colour means;
  • apply the rules for the user’s actual municipality;
  • show what belongs in the bin, what does not, and common mistakes;
  • present the answer in the user’s language; and
  • say “unknown” when the evidence is insufficient.

That last state is deliberate. No answer is safer than confidently applying another city’s rules.

Waste meanings are not embedded in the vision model. The model learns physical forms such as a wheelie bin, igloo container or textile bank. A versioned region pack then maps form, colour and jurisdiction to locally sourced disposal guidance. Adding another city is primarily a verified data contribution rather than a model retraining exercise.

Devices without a suitable rear camera receive a rules and registry interface instead of a broken camera experience. The PWA also caches the rules browser for offline use, although scanning requires a connection because inference runs on the server.

Frames are processed in memory and discarded. Nothing is retained unless a future contribution flow obtains explicit consent for that individual frame.

How I built it

The client is an installable React and TypeScript PWA built with Vite. It includes a real camera pipeline, accessibility-conscious components, right-to-left layout support, offline region data, URL-addressable application states, and explicit network, permission,n and uncertainty states.

Before sending a frame, the client applies four load-bearing gates:

  1. motion detection;
  2. a four-frames-per-second limit;
  3. a result lock that stops redundant requests; and
  4. a twenty-second abort.

These gates protect both the user experience and the project’s zero-cost infrastructure target.

The backend is a FastAPI and ONNX Runtime inference service supporting both REST and WebSocket transports. Its pipeline is:

  1. a validator finds bins;
  2. detected regions become crops;
  3. an identifier classifies their physical form;
  4. colour is measured from the pixels;
  5. the regional resolver produces the local answer; and
  6. disagreement between the two models flags a possible unknown bin type.

The web client and Python service share byte-for-byte protocol fixtures so that a change on one side cannot silently break the other.

The ML pipeline is written in Python and runs training through Kaggle. The current validator dataset contains 18,954 images: 1,480 positive bin images and 17,474 background images. Splits are group-aware so photographs of the same physical bin cannot leak across training and evaluation.

Model artefacts carry sidecars containing their classes, input shape, normalization, measured accuracy, latency evidence,ce and ship-gate verdict. The service refuses to load an artefact unless that sidecar explicitly says it may ship.

Challenges I ran into

The hardest problem was separating a bin’s appearance from its local meaning. A blue lid is not a universal label. Treating colour as a class would reproduce the original prototype’s biggest limitation, so I separated physical recognition, measured colour, and municipal rules into different layers.

The second challenge was building for a zero-cost pilot without pretending compute is unlimited. Multiple bins mean multiple inference passes, while multiple users compete for the same CPU. I built client-side request gates and a server-side degradation ladder that first lowers frame rate, then switches users to tap-to-scan, and finally returns a clear retry delay instead of allowing the service to collapse silently.

Cloud experimentation also produced several unexpected failures. A Kaggle image reported CUDA as available while allocating a P100 GPU that its bundled PyTorch build could not use. Requesting a T4 explicitly and checking its compute capability before downloading the dataset made that failure fast, explainable,e and reproducible.

The most important challenge appeared after the first validator completed. It scored 0.7524 [email protected] in fp32, but its int8 ONNX export scored only 0.025. The deployment gate rejected it. That was frustrating, but it demonstrated why the gate exists: without it, the project could have deployed a model that looked healthy while returning noise.

The current quantization probe is testing numeric formats, calibration strategies, preprocessing, ng, and mixed-precision boundaries. Until a configuration passes the fixed accuracy budget, production deployment remains intentionally blocked.

Accomplishments that I’m proud of

I am most proud that this is no longer only a model demo. The project now has:

  • an installable and responsive PWA;
  • a real camera capture and request-gating pipeline;
  • a production-shaped FastAPI inference service;
  • REST and WebSocket transports with a shared wire contract;
  • a multilingual, jurisdiction-aware rules resolver;
  • an offline rules experience;
  • a reproducible dataset and Kaggle training pipeline;
  • automated accuracy, latency and artefact ship gates;
  • a load-shedding and degradation strategy; and
  • extensive automated tests across Python, TypeScript and the service boundary.

The trained validator also achieved 0.9793 frame-level specificity across 2,662 background test frames, exceeding its 0.97 negative-corpus target. This matters because the predecessor could hallucinate bins in ordinary images.

Most importantly, the system refused to deploy its own defective int8 export. Treating a failed safety gate as useful evidence—rather than weakening the gate—is one of the project’s strongest technical accomplishments.

What I learned

I learned that a good ML product is much larger than its model.

A PyTorch score is not evidence that the graph actually served in production is accurate. Exported and quantized artefacts must be evaluated independently.

Performance numbers are meaningless without the hardware, split, and runtime that produced them. A CPU quota is not a guaranteed CPU allocation, and a passing unit test does not prove that a load-shedding path is reachable under real concurrency.

I also learned that uncertainty is a product feature. A carefully designed “unknown” result protects users, creates a useful contribution path, and is more trustworthy than a confident guess.

Finally, local civic information must remain traceable to its authority. A computer-vision model can identify a physical object, but it should not invent a municipality’s disposal rules.

What’s next for Smart Bin Recognition

The immediate next step is to complete the quantization investigation and produce a validator artefact that passes the unchanged accuracy gate.

After that, I will:

  1. complete the human form-factor adjudication pass;
  2. train and evaluate the identifier model;
  3. deploy the service privately on Cloud Run;
  4. require an authenticated health check to succeed while anonymous requests receive 401 or 403;
  5. connect the web client only after those deployment checks pass;
  6. finish and independently verify the Deggendorf region pack;
  7. complete the German and Arabic translations, followed by the remaining launch locales; and
  8. run a small field pilot with international and newly arrived residents.

The long-term goal is a free, anonymous public utility whose geographic coverage improves without selling user data, retaining private camera frames, or placing verified disposal guidance behind a paywall.

Built With

  • docker
  • fastapi
  • github-actions
  • google-cloud-run
  • hugging-face-hub
  • json-schema
  • kaggle
  • onnx-runtime
  • progressive-web-app
  • pytest
  • python
  • react
  • rest-api
  • service
  • typescript
  • ultralytics
  • vite
  • vitest
  • websockets
  • workers
  • yolo11
Share this project:

Updates

Submission history