Soft Mode (#32): Sculpting Intelligence - CLAI [Clay]

• 

  demo day

• GIF

  sculpting intelligence

Inspiration Our task was to design embodied objects for interacting with AI. Inspired by programmable matter and the Tangible Interfaces group at MIT Media Lab, our team Soft Mode took a different approach. What if you took a seemingly primitive material and gave it intelligence? Our medium: Clay.

What it does Clay becomes a physical medium for ‘sculpting’ digital content through an LLM. A material prompt language, where code is sculpting and ideas become matter. When a user interacts with a ball of clay, through embedded sensors and computer vision, the user is able to interact with generative mediums (image, music, video) using AI to translate physical action with the clay onto the digital content.

How we built it We embedded sensors into the clay and interaction surfaces and employed computer vision. We used open CV and

Challenges we ran into Clay is a wet, organic material posing challenges for embedded hardware such as sensors and training an LLM to understand an organic form.

Accomplishments that we're proud of A coherent working physical/digital prototype that conveys the interaction with the clay and translation through AI into a UI.

What we learned How to develop a new programming language through material and form.

What's next for Soft Mode: Sculpting Intelligence - CLAI [Clay] Seeking collaborators and grant opportunities for development.

Built With

• api