WalTer - Your Everyday Inventory Companion

WalTer is an autonomous inventory robot that navigates around your desk and keeps track of what you have and what you can do with it. Just find a messy desk and WalTer does the rest!

Comment

Inspiration

Diving into HackPSU, we had no idea what to make. We had buckets of hardware ready to go and splayed out all over the table. However, a tale as old as time, we didn't know what to create. While ideas were being thrown around, we wished there was a way to create project ideas with all the components we had. Suddenly, we had an idea: what if we made a robot that could take photos of all these items and use Google Gemini to generate project ideas?

What it does

Meet WalTer. WalTer is a personal inventory management system. All you have to do is place WalTer amongst the clutter of your desk and let him go! WalTer will take photos of each item on your desk and create an online inventory for you. From this inventory, you can select specific components and generate project ideas and instructions based on what you want to learn.

How we built it

WalTer Bot:

  • An Arduino Uno R4 is used to control the drivetrain, ultrasonic (distance) sensor, and OLED eyes. The ultrasonic sensor detects an item on the desk using a change in the distance from the sensor to the ground.
  • A Raspberry Pi Zero 2 and a Raspberry Pi Camera were used to take photos of each items. These items are then converted to a .jpeg image to place on the website.
  • A small screen was used to give WalTer a personality. After all, WalTer is your companion.

WalTer's Website:

  • Website to display a catalogue of items WalterBot has found
  • Identifies item using Google Vision image processing
  • Removes duplicate images of already catalogued items
  • Projects page where you can define projects using specific components WalterBot has catalogued, and use an LLM (google gemini) to develop ideas for projects using those item
  • Ask the website for advice and instructions on your project

Challenges we ran into

One main part of WalTer is categorizing the images WalTerBot uploads (identifying the object). We use google vision image processing for this, and while this is great at detecting what the object is broadly, it struggles with finer details like make or model. More advanced AIs that can handle this require payed plans, so we weren't able to implement them, meaning the image detection capabilities are restricted. Future versions with funds would enable WalTer would be able to identify objects with much greater accuracy and improve its effectiveness.

For the backend of the website, we had to use different tools and API's for image identification and for the project conversations, because of the limited nature of the free plan. Took a lot of time to set up and debug some of these.

Accomplishments that we're proud of

This was our first time using the Raspberry Pi Zero's DIO pins. This was very enlightening as it was vastly different from doing DIO on an Arduino.

What we learned

We learned lots about web design and how it interacts with IoT devices.

What's next for WalTer - Your Everyday Inventory Companion

More than just creating an inventory, we vision WalTer to be a personal desk assistant. In future iterations, WalTer could pick up items and organize them. Additionally, WalTer would have the ability to retrieve/find components as requested.

Share this project:

Updates