Inspiration
We felt that current grippers in industry specialize in overly narrow tasks, so we searched for alternatives that are more flexible and capable of a wider variety of tasks. Another issue that current industrial grippers have is in tactile sensing. Some grippers are incapable of distinguishing between delicate and hard materials, meaning that certain grippers could destroy fragile objects. After some research, we came across soft grippers, which are capable of grasping a variety of objects, and considering the hardware and time constraints for this hackathon, we felt that it would be a suitable solution to Ford's gripper challenge.
What it does
The soft gripper is pneumatically actuated, meaning that it is inflated with an air compressor to grasp objects. The grippers are made of silicone shaped by 3D printed molds we designed. Using a 12VDC compressor, we are able to fill a gripper with air, letting it grasp objects. Turning the compressor off will release it. Using a pressure sensor, we know when to stop the air compressor from inflating the gripper once an object is secured and prevent over-compression, which could lead to popping. We also have a camera to attach to the UR10E robot arm, which can capture images of the object and its scene. Using gaussian splatting, our program reconstructs the scene, generates a point cloud, determines the object's location, and plans motion toward the object for grasping.
How we built it
We iterated through many designs of the soft gripper, inspired by research papers we found online Design of a Versatile Soft Pneumatic Gripper. We ended up using a solution of a star-shaped silicon molded gripper sealed using fabric.
For gaussian splatting, we used InstantSplat, which can train the splats (which are basic objects used to "paint" a 3D map) with PyTorch much faster than with traditional gaussian splatting (which performs training on a pixel level with neural radiance fields). InstantSplat also contains submodules for structure from motion, which provides us with point cloud maps. Open3D lets us process point cloud files to determine optimal grasping locations.
Challenges we ran into
For gaussian splatting, we initially tried to use a Jetson Orin Nano as compute. Its Jetpack SDK caused several conflicts with InstantSplat that we spent hours resolving. When we were finally able to try it, it did not have sufficient memory for InstantSplat to run. We ended up borrowing a Lenovo Legion to use as compute. We spent more time setting this laptop up for InstantSplat, but when we finished doing so, we got great results. We also struggled with finding methods to localize our camera, so we ended up deciding to just use ground truth values from the robot arm position. The main issue we had with the pneumatic gripper was with leakage. We had to be very careful with removing the gripper from the molds as to not create any cracks or crevices air could escape from. However, to actually interface with our pneumatics system, we had to puncture a hole into the center of the gripper and attach a tube. This created many problems for us, as air would keep leaking out of this hole no matter how much adhesive we added to seal it up.
Accomplishments that we're proud of
InstantSplat is an incredibly impressive project, so we are very proud to have it working with the rest of our algorithms. Being able to render a scene from just a few images and construct a robust point cloud map is very satisfying, so we're happy to integrate these feats with grasp planning for soft grippers. Soft grippers themselves are also a very creative approach to the grasping problem, so we are very proud to have implemented them so smoothly given the time and hardware constraints.
What we learned
We learned a lot about the research process, especially in the field of robotics. Finding existing work and adapting it to new applications is a valuable way to approach many problems. Gaussian splatting is an incredibly deep field, and although we only scratched the surface in 36 hours, what we've learned has made us curious to discover more.
What's next for Gaussian Splatting for Soft Gripper Grasp Planning
InstantSplat only performs scene reconstruction after all images have been captured, which is an offline process. If we were to research deeper into online methods for scene reconstruction as new images are captured, our project would run even faster. We also aren't able to directly connect our own program to the robot arm in this hackathon for security reasons, but in a lab setting, we would be able to control all components of this project from one program. This would also provide us with better pose estimates of our camera from the robot arm. For the soft gripper, some of our hardware components malfunctioned. Initially, we planned on using a microcontroller to communicate between a 3-way valve and the compressor to allow the gripper to maintain a gripping state and an idle state. When we wanted to grip, the compressor should turn on until an internal pressure sensor passed a threshold, switching the valve to a hold state. When we wanted to release, the valve would switch again to an open state, allowing the air inside the gripper to escape and release the object. However, our 3-way valved did not work properly, forcing us to give up on this idea.
Log in or sign up for Devpost to join the conversation.