Inspiration

Throwing away trash is an afterthought until you lose the ability to do it. Every year in the US, there are 5 million adults (about 1 in 7 people) who live with mobility disabilities. Accessibility often stops where waste management begins, and after one of our team members recently broke his foot, we realized that helping him navigate a long dorm hall to throw away a piece of trash is something we can easily do. However, instead of helping people navigate to stationary bins, we realized, why not have the bin navigate to them?

What it does

Therefore, by combining AI with multimodal navigation, our project, “Litter-ally Trash,” allows users to summon a moving trash bin via voice commands, hand gestures, or simple keyboard inputs to throw away waste in a quick and accessible manner.

How We Built It

To design "Litter-ally Trash", we focused on providing responsive navigation and an accessible UI. Our first mission was to make a user-friendly drive base so our trash cans could navigate towards their requester. This required network programming over wifi and Bluetooth since typical users would not have access to a computer connected to our machine. In addition, we taught ourselves how to set up an Arduino to power Mechanum wheels and prepared our machine to travel in any direction. Some examples include navigation via voice commands with Grok, hand movement with Ultralytics’ “Yolo” model, and Photon messaging to make talking with our agent on the bin robot more personal. Next, our group focused on building the infrastructure to help users throw their trash in the right bin. This involved training models using OpenAI's CLIP to detect whether trash belonged in the compost and setting up servos so the trash can lid opens and closes correctly. Finally, to make the setup easier, our group created a web app accessible on a phone and computer so users could access all the commands without the command line. This utilized Tiger Data to help users see what they've recycled and thrown and how they can continue to be sustainable.

Challenges We Ran Into

For this hackathon, our team's biggest hurdles were likely transitioning to hardware for the first time, fixing power and torque issues on the motors, and ML bottlenecks. To start, we struggled to learn how to use our hardware and build the drive base since many of us had no robotics experience prior. Additionally, once we mounted the motors and set the batteries, we immediately had trouble getting them to spin in the right direction. In addition, Pis and Arduinos were new environments, so we spent a couple of hours learning how to connect remotely and write code that could be received and sent from the mini computers to our main laptops. Finally, the last challenge our team faced was ML bottlenecks, since we wanted this application to be responsive. As such, when the camera was taking ages to render the image, we had to change strategies and plan out how to integrate our GPU for the task. In addition, this required us to simplify and optimize our code to have lower network latency.

What We Learned

We were grateful to have the experience in learning more about embedded programming and had a ton of fun doing network programming to send commands to our pis and learning how to create a movable vehicle from scratch.

What's next for Litter-ally Trash

Our next steps for Litter-ally Trash are adding asynchronous commands such that the user does not need to control our robot, and that the robot can go travel around obstacles and meet up with requester. In addition, we also would like to implement heat sinks and some sort of cooling on the bot because we have dealt with persistent heat issue.

Built With

• arduino
• clip
• css3
• cursor
• gemini
• godaddy
• grok
• html5
• javascript
• openai
• photon
• python
• raspberry-pi
• tigerdata