omnigrip

Inspiration

For the Ford Grasping track, we focused on one of the core bottlenecks in modern robotics: robots are getting better at seeing and moving, but reliably grasping different real-world objects is still hard. In manufacturing environments, objects vary in shape, material, orientation, and pickup method, which makes grasping a major blocker for flexible automation. OmniGrip was inspired by that gap. We wanted to build a system that improves a robot’s ability to decide how to grasp an object, not just detect that it exists.

## What it does

OmniGrip improves robotic grasping by connecting visual perception to object-specific pickup strategy. It detects parts in a live camera feed, identifies the best target, and outputs a grip plan based on that object’s geometry and material properties. For metallic parts like a bearing, the system routes the object to an electromagnet-based grasping workflow and provides the placement and approach data needed for pickup. We also built an ESP32-powered magnet controller with a browser-based interface so the grasping hardware can be triggered wirelessly and safely.

## How we built it

We built OmniGrip as a combined perception-and-actuation prototype. On the software side, we used Python, OpenCV, and Ultralytics YOLO for real-time object detection and target tracking. We paired that with a CAD-informed grasp analysis pipeline that evaluates object geometry offline, computes grasp candidates, and stores those results in a cache for fast live use. On the hardware side, we used an ESP32 with ESP-IDF and C to drive a relay-controlled electromagnet, plus a lightweight Wi-Fi web dashboard for testing and control. This let us demonstrate an end-to-end grasping improvement: see the object, reason about how to pick it up, and activate the appropriate hardware.

Challenges we ran into

The main challenge was that grasping is not just a vision problem and not just a hardware problem. We had to bridge multiple layers: detecting objects reliably, keeping tracking stable across frames, matching detections to supported CAD assets, and converting that into an actionable grasp plan. Another challenge was building something that reflects real manufacturing constraints, where you need consistent behavior and safe actuation, not just a one-frame demo. We also had to work within limited time while only some object classes had fully prepared CAD and grip-planning assets.

## Accomplishments that we're proud of

We’re proud that OmniGrip directly addresses the Ford track’s core problem: improving a robot’s ability to grasp objects. Rather than stopping at classification or detection, we built a prototype that links perception, grasp reasoning, and physical control in one pipeline. We’re also proud that the system is modular and scalable. The same architecture can support additional part types, gripper strategies, and robot platforms, which aligns well with Ford ATP’s “build once, scale everywhere” mindset.

What we learned

We learned that useful grasping solutions come from integrating AI/ML vision with practical robotics constraints. Detection is only the first step; real value comes from selecting a feasible grasp strategy for the object in front of you. We also learned how powerful CAD-informed reasoning can be in manufacturing contexts, where known parts can be analyzed ahead of time and used to make grasping more reliable. Most importantly, we learned that improving grasping requires coordination across software, geometry, and hardware, not isolated optimization in one area.

## What's next for OmniGrip

Next, we want to expand OmniGrip beyond a single supported metallic object and add more part classes, more CAD assets, and more gripper types such as suction and mechanical gripping. We also want to connect the system to a robotic arm so the output becomes full robot motion instead of just grip planning and magnet control. Long term, the goal is a flexible manufacturing grasping platform that can generalize across many parts and environments, making robotic systems more versatile, reliable, and useful in the kinds of industrial settings Ford cares about.