Inspiration
We do the R&D, so you can stay on DND! (Do not disturb)
Some of our group members have personally woken up in the middle of nights, stumbled around from place to place, and wasted a lot of time resetting our 3D printers. We want to automate 3D printing. But much more than that, we envision a world where robots build robots.
What it does
Introducing PrintDND, a robot that is able to autonomously clear the printers from their previous job and place the print down in a desired location. This allows printing to occur while you can be on Do Not Disturb.
How we built it
This robot was provided by one of the Hack the North sponsors, BracketBot. This robot has a bimanual 7-DOF setup with a mobile 2-wheel base. The robot uses reinforcement learning to balance itself upright.
We took this hardware and software stack provided by BracketBot and evaluated the problem via first principles. We broke down the rudimentary movements required to remove prints from the printer and collected data via teleoperation with the Meta Quest 3S. We collected over 100 episodes that we stored in BracketBot's cloud service.
This data was then audited for quality and used to fine-tune the Physical Intelligence PI0.5 Vision Language Action (VLA) model. Once the model was trained, we built upon the deployment stack to use the model for inference.
On top of training the model to reset the 3D Printer, we also implemented voice prompt. The BracketBot will detect "Hey Bracketbot" and it will listen and do the command that we tell it to. Example, "Hey BracketBot, reset the printer". The BracketBot is also able to reply to our command.
Challenges we ran into
Network and connectivity has been a huge issue for us (HTN, eduroam, and even starlink wifi networks were spotty and inconsistent). This involved us moving our entire setup 3 separate times. Each time, it seemed like the problem was slightly different. Whether it was trouble pairing with the meta quest, uploading to the cloud, or just connecting via ssh, we met a broad range of network issues.
Data collection was also buggy, deleting random takes, recording at the wrong times, and sometimes not recording at all (75% of our recorded data got lost in the Bermuda Triangle). */- We realized that there is a local storage folder called /pending, which holds the recordings until the bot is able to upload them to the cloud. It turns out that /pending has a max limit and begins to overwrite files after the limit. We realized this too late! Much of our data was overwritten when we were recording without wifi.
Lastly, the speed of the VLA proves to be a big issue. It takes a lot of compute to plan all the joint angles of this robot. This was largely counteracted by the fact that we had Montreal servers doing the inference. However, the robot still moves incredibly slowly. As we write this, we are investigating the chances of getting a Toronto or Waterloo server to speed things up!
Accomplishments that we're proud of
Having a successfully trained VLA model that recognizes object to remove from the 3d Printer, and how to flex the base plate. We built a custom 3D printed jig that minimizes the force requires to flex the build plate (and shake the print off the bed). We thought this was a clever solution to the grippers having a maximum lift weight of 2kg, which is much less than what is required to lift the magnetic plate normally.
Training the VLA proved difficult, as small things such as the home position coordinates being similar to how we lift the build plate easily confused the robot into thinking it had already accomplished a task right off the bat. Additionally, the 3d printer perfectly occludes the head camera from seeing the print, thus requiring our VLA to rely heavily on the wrist cameras. This introduces a problem that depth is hard to perceive with just one camera. Since it is a VLA and not hard-coded, the robot will attempt to retry grasping the part, even if it misses it the first time!
What we learned
Find a way to WFH for reliable wifi! Jokes aside, we learned a lot about VLAs and how robotics control works! The coolest part was working all night with the BracketBot staff, pulling our hair out and bonding over the shared experience of trying to make the robot work. We learned that data is everything, and the importance of varying our data so our model is generalized and not overfitted.
The importance of having a good team was made clear to us. No one likes working on a couple hours of sleep, but when the team is having fun and the morale is high, even working with bad wifi can be fun (kinda). Super big shoutout to Isaac and Jason from BracketBot for going the extra mile with helping us debug our software issues. And big thanks to Brian and the rest of the BracketBot team for going above and beyond in ordering starlinks for when our wifi wasn't working, encouraging us, and helping us out!
What's next for PrintDnD
- Iterate a process for the entire 3d printer workflow, including the door, replacing filaments, and diverse print geometries.
- Include a navigation to go from printer to printer
- Include dynamic height detection to handle different printer setups
- Find ways to speed up the robot!
- Expand from print removal to full on robot assembly!
- Experiment with an exponential moving average (ema) to make movement smoother!
NOTE: we bought a domain off GoDaddy "printdnd.ca", but .ca domains take about 24 - 48 hours to certify. Our website is hosted at https://printdnd.vercel.app/ for now!
Did you know? PrintDnD's name was inspired by a prior startup by Makis and Jon called PrintBnB, a 3D printing marketplace!
Built With
- bracketbot
Log in or sign up for Devpost to join the conversation.