Inspiration

Putting the wrong item in a recycling bin can spoil a whole load, and most people don't know which bin a greasy box, a crushed bottle or an old charger belongs in. Rules also change from place to place. We wanted something that answers the question at the moment you're holding the item: what is this, and where does it go?

What it does

EcoScan reads waste through a camera and tells you what each item is, its material, whether it's recyclable, which bin it goes in, how to prepare it, and ideas to reuse it.

  • One command: npx ecoscan opens a terminal app. /waste scans with your camera or an image file.
  • Local rules: /region <place> makes the bin names and rules follow your area.
  • Several items at once: each item gets a numbered, colour-coded box on the saved photo.
  • Stream: /stream watches the camera live like a recording and sorts items as they appear. It pauses when nothing is in view and starts again when something does.
  • Impact view: /stats shows what you sorted, what needed special disposal (hazardous and e-waste) and how many reuse ideas you got.
  • An SDK: the same call works inside other apps (npm i ecoscan).

How we built it

  • TypeScript with Node 18+, published as one npm package that holds both the SDK and the terminal app (Ink and React).
  • Groq's vision API (default model qwen/qwen3.8-27b) returns structured JSON, checked with zod. Bad replies are retried, and one bad item doesn't break a whole answer.
  • ffmpeg is bundled, so there is nothing else to install. It does the camera capture on Windows, macOS, Linux and Raspberry Pi.
  • Boxes and labels are drawn in pure JavaScript, with no native image libraries.
  • Stream is our own pipeline built around the model's latency. It skips frames where nothing changed, picks the sharpest frame in each time slot to avoid motion blur, and packs several moments into one numbered image so one call covers about two seconds. Still pictures are cut into four full-resolution tiles. Old frames are dropped, never queued, and it slows itself down when Groq rate-limits it.
  • An eval script scores accuracy against your own labelled photos.
  • A landing page built with React and Vite.

Challenges we ran into

  • Box coordinates: the model measures its 0 to 1000 box scale against the image's longer side on both axes. We found this by testing real captures, because our first guess drew boxes in the wrong place.
  • Phantom items: the model sometimes listed a face or the same bottle twice, so we added prompt rules and a filter.
  • Real-time limits: a model call takes a second or two and free tiers have rate limits, so we couldn't classify every frame. Counting also double-counted the same item across scans, so Stream reports sightings, not exact counts.
  • Cameras: Windows lists virtual cameras first, cameras get locked by other apps, and capturing a two-monitor desktop squeezed the picture too small. Each needed fixing in testing.
  • Honest numbers: the model reports about 95% confidence nearly every time, so we show it as the model's own estimate, not as accuracy.

Accomplishments that we're proud of

  • A working tool anyone can start with one command, published on npm.
  • One package that serves both people (terminal) and developers (SDK).
  • A live camera mode that stays responsive under rate limits and avoids wasted calls.
  • A built-in way to measure accuracy, so we can quote measured numbers and not guesses.
  • A clear list of what we have and haven't tested.

What we learned

  • For live vision, the hard part is deciding what is worth sending to the model, not calling it.
  • A model's self-reported confidence isn't accuracy. Measure it against labelled photos.
  • Counting from video is an estimate unless items can be tracked, so we say "sightings".
  • Small details like which camera is selected decide whether a demo works.

What's next for EcoScan

  • Add a small model that runs on the device to detect when and where an item appears, so the cloud model is only used to name it. That would cut cost and rate-limit problems.
  • Publish a measured accuracy number from a larger labelled photo set.
  • Test on macOS, Linux and Raspberry Pi, and build toward smart bins, kiosks and a mobile app. These are planned and not tested on hardware yet.
  • Improve counting with real item tracking.

Built With

Share this project:

Updates

Submission history