Artificial Gluten Intelligence

3d gaussian splat
yummy

Robotic Sandwich Maker that makes a BLT that's 100% edible (and tastes good!). [1]

Language models are exciting! But how well does "intelligence" actually translate to the physical world?

Artificial Gluten Intelligence is an autonomous bimanual setup of SO-101 [2] arms for an end-to-end Video Action Model [3] to construct a sandwich. We follow a supervised training method for behavior cloning. In addition, we create a 3d reconstruction and ablate training with and without 3d [4]. We get better performance with ground truth world model supervision but find stability issues we were not able to address in the limited time.

Please see attached video for a demo.

[1] https://devpost.com/software/artificial-sandwich-intelligence [2] https://huggingface.co/docs/lerobot/en/so101 [3] https://unified-video-action-model.github.io [4] https://real2render2real.com