Inspiration
Pharmacists and pharmacy technicians spend a large amount of precious time packing and filling prescriptions when they could be communicating with customers, insurance, or other medical professionals. With our project, they could easily task the arm to pack and fill prescriptions accurately and dedicate their attention to someone who needs their assistance.
What it does
LePillbot uses the Hugging Face LeRobot SO101 arm platform and computer vision through a mounted webcam to detect objects of interest in its area of operation, grab them, and sort them into their appropriate containers.
How we built it
We assembled the arm based on parts we 3D printed and some servo motors we bought. The AI model controlling the arm through computer vision was trained by a member of our team based on data that we collected and cleaned ourselves.
Challenges we ran into
- Calibrating the servos to operate properly was difficult.
- We had trouble mounting the webcam to the arm at first since the tape we bought to attach it was not sticky enough
- Training the AI was a challenge since we needed to use two types of data simultaneously, both the video data and servo motor position data.
Accomplishments that we're proud of
We successfully built an entire ML pipeline from ground zero. We started with zero data, and through our efforts managed to train and implement a model that successfully accomplished its given task.
What we learned
We learned a lot about how to collect data for computer vision and how to train robotic components. Both of these technologies are rapidly growing in the industry today and we are excited to learn valuable career skills through our project.
What's next for LePillbot
We would want to train the AI to detect differently shaped pills alongside colors. With our time constraints, we decided to focus solely on one differing characteristic of the pills, but with more time and budget we would be excited to expand the detection capabilities of the computer vision tech to make our project into a practical and affordable tool for pharmacists around the world.
