-
-
GIF
Using AR overlays on real clay for the perfectionist in you :)
-
GIF
Choosing pottery styles to make from around the world
-
GIF
An AI Artisan guide that helps you in guided mode
-
GIF
Carrying 3D scans of your cool creations with you everywhere and sharing them with the community
-
See our Behind The Scenes video: https://www.youtube.com/watch?v=AJUbhVCyTGY&t=8s
-
Meet our Team Electrocula, inspired by 1960s version of a VR headset!
Hello everyone! 👋 We are Team Electrocula, a group of six XR enthusiasts who are introducing Creta, a Mixed Reality (MR) sculpture and pottery app, as part of our submission for the 2024-25 Global XR Design Challenge.
Inspiration ✨
So how does a team of 6 strangers come up with a killer app idea for a 6 week hackathon? Discord calls, Figjam brainstorming sessions, sharing real life experiences, and lots of sticky notes. Several rounds of brainstorming and a couple hundred stickies later, we found a topic that resonated with us - preserving cultures, traditions and crafts. In a digital era, it can get harder to get in touch with more traditional mediums, however we wanted to augment and elevate the hands-on tactile experience that learning crafts thrives on. What if we could use new tech to expose people to the ways of the past through pottery? We can make it accessible at home without needing any specialist equipment, a rich history attached and a few of our team members have experience with pottery- meaning we could access a community of users to get insights from during user testing.
We conducted extensive research in order to prove market viability. In our competitive analysis we found that there are a few pottery and sculpture apps for Virtual Reality, but none in the Mixed & Augmented Reality space. The apps didn't help users transfer skills to real clay, and lacked repeatability. This meant there was a gap for a solution to help users learn but whilst creating a real work of art. We developed some early prototypes in Shapes XR to gather feedback from 8 users, half which had no experience with clay, and half who had some experience. From our survey results, 100% of responses indicated that our solution would have a positive impact on their sculpting experience. We realized that there was demand for a tool that enables higher precision sculpting, provides useful feedback, and enhances accessibility to clay pottery & sculpture technique for those without access to a studio.
Hence, Creta was born, which means 'clay' in Latin. Fun fact: our Team name Electrocula was inspired by the 60s version of a VR headset i.e a Wearable TV, covered by BBC Archives.
What it does 🏺
Our concept is a Mixed Reality learning experience that combines hands-on pottery creation with cultural storytelling. It guides users through crafting historically inspired pottery, like an Ancient Greek amphora vase, using real clay and AR overlays to provide step-by-step instructions, semi-transparent templates, and AI-powered feedback. Along the way, users immerse themselves in the history and traditions behind the craft, guided by an AI artisan mentor who answers questions in a culturally authentic style. The app also offers a sandbox mode for free-form creativity, a community shelf to share & appreciate creations and a personal shelf to carry their collection with them anywhere.
How we built it ⚙️
We used a range of tools to build our prototype, including:
1) FigJam as a team brainstorming and planning area, where we collaborated in our meetings starting from the design process to the implementation aspect of our app.
2) Figma as a UI tool to create UI assets and a cohesive design language for the app.
3) ShapesXR to create quick and effective prototype iterations to get user feedback. We created VR clay model purely for demonstrative purposes, so that anyone opening the prototype without actual equipment can also have the pottery experience.
4) Logitech MX Stylus integration with ShapesXR to support prototyping, simulate drawing functionality and stylus menus.
5) Unity to create our AR pottery demo app and Meta Llama 3.0 to integrate Large language model capabilities in our AI artisan mentor. We also utilized the passthrough, surface tracking, and hand tracking capabilities of Meta Quest 3, with Meta XR Core SDK & Meta XR Interaction SDK implementations.
6) Blender to polish our 3D Models, & Adobe After Effects to bring together all our efforts in the Trailer and Extended videos.
Challenges we ran into 🤺
1) Time Zone Management: As a global team, spanning from California, Indianapolis, Glasgow, Massachusetts, New Hampshire, England, and Tel Aviv, one of us was always awake somewhere. This made coordinating meetings and collaboration across different time zones challenging. To overcome this, we’ve used this to our advantage, by using a well-maintained Discord channel, FigJam, Notion, Google drive and twice-weekly meetings and workshops, we’ve been able to make steady progress with close collaboration!
2) Skillset Integration: With each team member specializing in different expertise (from research to UX design, development, visuals) integrating diverse skillsets into a cohesive product was complex. We addressed this by defining clear roles early on, based on expertise and growth areas while fostering cross-discipline teamwork during brainstorming and prototyping phases.
3) Polish with limited time: Building a functional and polished prototype within six weeks required lots of prioritization. By focusing on key features which brought the most value to our users, like the guided tutorial and AR overlays, we ensured the core experience was robust and impactful. We initially intended for the AI guide to be dressed as an artisan from that era, but we were unable to customize it due to Meta Avatar integration limitations. We tried to overcome this by immersing the users in other ways, such as with a relevant 3D environment and audio representing that culture and time.
4) Selecting a concept: There were many traditional crafts that we were keen to showcase in our demo, but due to time and resource limitations, we had to stick to one. Since some of our team members were familiar with pottery, and also had links to the community that would allow us to get user feedback, we identified it to be our chosen craft.
Accomplishments that we are proud of 🎉
1) Cohesive Collaboration: Successfully bringing together a global team to ideate, prototype, and refine a unique MR solution for preserving cultural traditions.
2) User-Centered Design: Conducted multiple rounds of user testing, validating our approach, and creating several iterations of our prototypes, to ensure that the app met the needs of pottery enthusiasts.
3) Innovative Concept: Delivered a MR-based pottery experience that bridges hands-on learning with cultural storytelling, standing out as a unique and impactful idea in the XR space.
4) Accessible Craftsmanship: Created an app that makes pottery more accessible, removing barriers like cost, intimidation, and lack of knowledge behind techniques, such as historical or cultural knowledge.
5) Polished Prototype: Developed a functional and visually appealing prototype showcasing key features and interactions, ensuring that we met the judging criteria throughout this process. ➡️ Don't forget to check out our ShapesXR full walkthrough video and a highlight reel!
6) Cultural Preservation: Successfully integrated storytelling and history into an engaging learning experience, supporting the preservation of ancient craftsmanship in a modern, interactive way. This way we could ensure that although our target audience is somewhat niche, the impact on them is high.
What we learned as a team 📚
1) Balancing Scope with Feasibility: Initially we brainstormed many ambitious features, from creating expansive craft library to gamification. However, we quickly learned the sheer importance of prioritization, focusing only on core functionalities like AR-guided tutorials and cultural storytelling aspects. We created a google form to get feedback from potential users and got encouraging feedback for the first iteration of our app, and went on to implement suggestions in the second version as well. By narrowing our scope, we ensured we had a polished prototype while keeping future expansion in mind.
2) Merging Technology with Tradition: Bridging the hands-on, tactile nature of pottery with digital AR overlays posed as a unique challenge. Hence we learned how to blend the two effectively to enhance the experience, without overshadowing the craft itself. This taught us how technology can augment, rather than replace, traditional skills and methods.
3) Crafting a Story-Driven Experience: From designing pottery inspired by historical artifacts to crafting a narrative voice for the AI mentor, we learned how to create an educational yet engaging experience that celebrates traditions while making them accessible to modern users.
What's next for Creta ⏭️
The future of this app could envision a comprehensive platform for preserving and celebrating global craftsmanship. Users could explore an expanding library of crafts and techniques from diverse cultures, like Glass Blowing and Dhokra art, each with its unique historical context and traditions. Advanced AI could enable personalized learning paths, adaptive feedback, and collaborative features for group workshops, where users could join classes with others, to enhance social interaction despite being in different locations.
The app could integrate MR-enhanced community challenges, gamification elements such as leaderboards, and virtual exhibitions to foster a global network of craft enthusiasts. By partnering with cultural organizations and museums, it could evolve into an educational hub, bridging the gap between ancient artistry and modern technology while inspiring new generations to keep these traditions alive.
Credits 🌟
Big Shoutout to our sponsors, Immersive Insiders, Meta, IDEO & Logitech!
Sources used: Link
Check out our exclusive Behind the Scenes Video:
Extra Goodies:
In our Google Drive Supplementary folder, you can access
- Sculpture Demo Quest APK
- Llama 3.0 Chat APK
- ShapesXR 60 Second Highlight Reel.mp4
- ShapesXR Prototype Full Walkthrough Video.mp4
- Trailer Video.mp4
- Extended Video.mp4
- ShapesXR Space Links and Codes
Try Out These Links:
ShapesXR Prototype Spaces
Please Note: All subsequent spaces are linked to PART 1, use Code: 265ez32e to begin viewing the ShapesXR prototype.
- PART 1: Users start the application, setup a 'safe space' on a designated surface, and select a lesson
- PART 2: Users follow a step-by-step guide with the help of a Holonote-driven mentor to learn how to make their selected pottery model.
- PART 3: Users use the Logitech MX Stylus to draw on their built pottery model, and later carve this pattern out using a knife
- PART 4: Users complete the lesson, and can create a 3D scan of their creation as a way of sharing their work with the Creta community.
Sculpture Demo Quest Apk: Link
Figma Creta UI Assets: Link
FigJam Brainstorming Board: Link
Log in or sign up for Devpost to join the conversation.