SortWise AI

• 

  sortwise

Inspiration

We often throw things away without really knowing where they should go. Is this container recyclable? What material is it made of? Should an old electronic item go with regular waste?

Even when people want to manage waste responsibly, figuring out how to sort different items can be confusing. We wanted to explore how AI could make this everyday task easier and help people make more informed decisions.

This is how we came up with SortWise AI — a tool that turns a simple picture of a waste item into useful information about its material, category, and possible disposal options.

What it does

SortWise AI helps users identify waste by uploading an image. It identifies the likely item, classifies it into a waste category, and provides disposal guidance, preparation steps, and relevant warnings.

It supports plastic, paper and cardboard, metal, glass, organic waste, general waste, electronic waste, and unknown items.

Instead of only telling users what an object might be, SortWise AI aims to help them understand what to do next. If the system cannot identify an item reliably, it can handle uncertain results rather than forcing every image into a category.

How we built it

We built SortWise AI using Python Fast Api and Next JS, with pretrained multimodal vision models accessed through OpenRouter. We also included Google Gemini as an independent external fallback and TrashNet as a local classification fallback.

The system analyzes uploaded images and organizes the results into structured information, including the item name, material, category, confidence score, visual evidence, disposal guidance, preparation steps, and warnings.

An important part of our design is separating AI predictions from disposal decisions. Application-level validation handles malformed or uncertain responses, while a deterministic Python decision engine controls the disposal guidance. This gives us more control over the recommendations instead of relying entirely on generated AI responses.

We use pretrained models rather than fine-tuning our own models.

Challenges we ran into

One of our biggest challenges was realizing that an AI response can sound confident and still be wrong. During live-image testing, we encountered cases where a plastic bottle was classified as metal and a glass item as paper or cardboard. This showed us that we could not simply trust every prediction.

Deployment was another major challenge. We struggled to deploy the backend on Vercel, so we had to explore alternatives and eventually try Hugging Face Spaces. Getting everything working involved trying multiple accounts, resolving numerous errors, and troubleshooting issues along the way. It took a lot of patience to get past these problems while working within the limited time of a hackathon.

We also had to consider provider availability, varying response times, malformed outputs, and images containing multiple objects. Although we added validation and fallback options, these cannot eliminate every classification error.

Finally, disposal rules vary by location. Our guidance is currently general, so providing reliable, location-specific recommendations remains a challenge we want to address.

Accomplishments that we're proud of

One thing we are especially proud of is that we didn't give up when things didn't work as expected. From struggling with backend deployment to dealing with countless errors, we had to try different approaches, troubleshoot problems, and keep moving forward despite the limited time of the hackathon.

We managed to bring our idea to life by building a working system that combines AI-based waste identification with practical disposal guidance. We also developed a backend with 56 passing automated tests and tested it with real images to understand how it performs outside ideal conditions.

Not everything worked perfectly, and we still have improvements to make. But we're proud that we kept trying, learned from our mistakes, and continued working towards our goal instead of giving up when things got difficult. For us, that persistence is one of the most valuable things we gained from this experience.

What we learned

This project taught us that building an AI application involves much more than connecting a model to a user interface. We learned the importance of validating AI responses, handling failures, using fallback options, and separating AI predictions from application logic.

We also learned that deployment can be just as challenging as development. When our backend could not be deployed on Vercel as expected, we had to explore alternatives, try Hugging Face Spaces, and work through numerous errors. This experience taught us to be patient, troubleshoot systematically, and adapt when our original approach does not work.

Testing with real images also gave us an important lesson: passing automated tests does not guarantee that an AI model will always make correct predictions in real-world situations. We need to evaluate its performance carefully and treat confidence scores and disposal recommendations with caution.

Overall, SortWise AI helped us understand that building a useful AI solution is an iterative process. It takes experimentation, debugging, testing, and continuous improvement—not just getting the first version to work.

What's next for SortWise AI

We want to evaluate SortWise AI more thoroughly using a labelled dataset of real waste images, so we can measure classification performance rather than relying only on individual examples.

We also plan to improve handling of visually similar materials, uncertain condition assessments, and images containing multiple objects. We want to make the fallback system more dependable and better understand how model availability and response times affect the user experience.

Another important goal is to introduce location-specific disposal guidance using authoritative local waste management sources and proper validation.

Our vision is to make SortWise AI a practical tool that helps people understand their waste, make better-informed disposal decisions, and take small steps towards a cleaner environment.

Built With

• ai
• antigravity
• codex
• fastapi
• generative-ai
• image-classification
• nextjs
• openrouter
• python