Inspiration
More than 1 in 4 U.S. adults lives with a disability, and the most common kind is mobility. Every year, 1 in 4 adults over 65 falls, often while reaching or climbing for something on a shelf. Memory is part of the problem too: about half of people on long-term medication don't take it the way it was prescribed, and when you can't remember which box something is in, finding the right item turns into guesswork.
Regular storage assumes you can reach, see, search, and remember. Labels and organizers still make you reach and read. A smart speaker can tell you where something is, but it can't hand it to you. The robots that really do fetch items, in hospitals and pharmacies, cost tens of thousands of dollars and fill a room. I wanted to build that same idea at the size and price of something that fits in a kitchen.
What it does
Index is a robotic cabinet that finds, fetches, and returns items for you.
- Ask by voice: say "I have a bruise on my knee." The AI agent checks what's actually in the cabinet, suggests bandages in compartment 1, and only moves after you say yes.
- Or use any phone: the robot makes its own Wi-Fi network and serves its own control page, so there's no app and no internet needed. You can search ("band" finds Bandages), tap to retrieve, rename compartments, and mark them empty.
- Retrieve: a 3-axis gantry moves to the compartment, extends a rack, and an electromagnet grabs the tray's steel washer. It pulls the tray out and sets it on top of the cabinet, within reach.
- Return: press Return tray or say "I'm done." It puts the tray back in exactly where it came from. If you ask for something else while a tray is out, it returns the first one automatically.
- Safe by design: the agent only offers what's really in stock (ask for aspirin and it tells you there isn't any), it never gives medical advice, and the robot echoes back every command (
GO 1→ACK 1). A misheard number gets caught instead of grabbing the wrong tray.
It has 15 compartments (3 × 5), and it works for anything stored in small bins: medicine, first-aid supplies, tools, lab parts.
How I built it
Hardware
- X (horizontal) and Z (vertical): stepper motors driving GT2 belts on linear rails, at 5 steps/mm, so positions are accurate to 0.2 mm.
- Y (depth): a servo-driven rack and pinion that pushes an electromagnet into the cabinet.
- Trays: 3D-printed, each with a steel washer for the magnet to lock onto. The cabinet sits on an aluminum frame.
- Homing: limit switches give the robot a fixed home position every time it starts.
- Electronics: one Maker-ESP32 board with on-board H-bridge motor drivers, powered by a 12 V supply.
Firmware (Arduino C++, ESP32)
- Non-blocking: a step sequencer means the motors, servo, magnet, web server, and USB link all run at the same time without pausing the motion.
- Switches: homing routine, debounced limit switches, and software travel limits.
- Teaching: I jog the arm to compartments 1, 5, and 11, and the firmware interpolates the other 12 positions. All positions are saved to flash.
- Return move: the tray goes back in 3 mm higher than the pickup height. Then the magnet turns off, the arm slides up 10 mm off the washer, and only then does it retract.
- Battery saving: the motors release when they don't need to hold position.
Control page: HTML/CSS/JS served by the ESP32 over its own access point, with live status, search, naming, jog controls, and teaching.
Voice: an ElevenLabs conversational agent with three client tools: get_inventory, retrieve_item, and return_item. A web page connects to the ESP32 over the Web Serial API, sends GO n, and only reports success when the robot replies ACK n.
Challenges I ran into
- Putting trays back was the hardest part. The tray caught on the opening, so I made it go in 3 mm higher. Then, when the magnet let go, friction dragged the tray back out with the arm. The fix was sliding the arm up 10 mm off the washer before pulling back.
- Limit switch bounce: the switches bounced when released, so the robot thought it had hit the same switch twice. I added debouncing and a re-arm distance.
- Mechanics: balancing one long linear rail on top of another, plus the Y axis, without wobble. I also slowed the X homing because the carriage shook when it hit the switch.
- Power: my first battery died in minutes because the motor drivers held full current while sitting still. The firmware now releases the motors when idle, and I moved to a wall supply.
- Keeping the voice page and the robot in sync: opening the USB port reboots the ESP32, and the page could lose track of which tray was out. I made every command ask the robot directly, so the robot is always the source of truth.
Accomplishments that I'm proud of
- A complete, working loop: voice request → confirmation → retrieval → delivery → return. It's shown live in the video.
- A three-axis machine that holds sub-millimeter positions and is taught by hand instead of hard-coded, so it can adapt to a different cabinet.
- An AI agent that can only act through verified commands, and a robot that confirms every one.
- About $300 in parts, compared with tens of thousands of dollars for hospital systems.
What I learned
How much real hardware differs from code that compiles. I learned stepper timing, switch debouncing, rack-and-pinion tolerances, power budgeting, and how to design a protocol between an AI agent and a machine so that a mistake gets caught before anything moves.
What's next
- Medication reminders ("It's 8 AM, here are your morning pills")
- Item counts with low-stock alerts
- Bigger, modular cabinets
- An optional caregiver view showing what was taken and when
