We started klurdy as a basic e-commerce store for African fashion and used data mining to onboard 100K+ products from the interwebs and we started getting loads of traffic. People would place orders via WhatsApp and work with local tailors in Nairobi to deliver the product. We realized there is usually an issue with the supply chain for fabrics and we would end up sending the customers a list of fabrics currently available and they would have to pick the fabrics that appeal to them. The challenge was the customer had to imagine the design they want with the fabric they pick and this might be the root cause of customer complaints where products ordered don't match what's delivered, We received multiple lawsuit threats from fashion houses whose products were on our platform without consent and that demonstrated that there was an issue with copyrights in the industry. We decided to ditch the standard image-based e-commerce site in 2019 and started learning and exploring the use of 3D in the fashion industry as we offer design services in Africa and Europe. We came across spark AR and it's amazing features in Jan this year and it had that WOW factor. The pandemic has not been kind to us as the majority of our design gigs came to a halt, having a massive impact on our revenue streams. We decided to go back to our roots, fashion, and use this hackathon to leapfrog our efforts to develop the MVP for our 3D experience. Due to social distancing, people can't go to the malls so we want to take the clothes to them by using AR/VR.
What it does
Klurdy is a PWA that allows end-users to visualize fashion designs with various materials that are currently available in the market. We upload custom 3D models of fashion designs and materials in our proprietary backend system and make them available in our easy-to-use product configurator. After the user has personalized the apparel to their liking, they can 'hang' it in their living room, office, or wherever they are through Instagram or Facebook stories. This allows us to take online shopping experiences offline, get feedback from friends or family. In short, we have developed an advanced hanger for any garment we choose to incorporate in our platform.
How I built it
As a design studio, we strive to be minimalists and find the shortest path to success. The web app is built with Angular and nest js to prototype the web-based experience and MongoDB as the database. We chose to use Babylon js as our 3D engine due to ease of use, supports many features and browsers. We integrated the web app experience with spark AR filters by deep-linking with the Facebook app. However, this did not work on IG due to deep linking being unavailable so IG filter was separate from the Facebook filter. This submission only includes the IG filter
The AR filter is built using js scripting, utilizing a multitude of features. The process used in IG filter is as follows:
- added a plane tracker to detect surfaces and added a null object as the child of the tracker. This is used for touch gestures.
- added a null object as the 3D model as a child of the null object. For this demo, we chose to use a simple t-shirt for girls with a minimalistic design. The 3D model loaded had different mesh parts that can be customized to offer different designs
- added textures for the various fabrics available. We added 4 African prints and 4 solid color textures, We then created standard materials for all of these textures and assigned default materials for the various meshes of the 3D model of the t-shirt. We digitized some African prints using adobe illustrator, which was a fun way to see how we can manifest some of our existing skills as a design studio
- we started scripting and fetched all the textures, materials, and mesh objects from the scene. We then created a native UI picker for selection options that the user can use to personalize the t-shirt. On monitoring selection, we apply appropriate materials to the meshes of the t-shirt.
- We monitor the touch gestures to move, rotate and scale the t-shirt appropriately
- added music to play in the background
All spark ar features were made using scripting to achieve better creative freedom.
Challenges I ran into
- Since the user should be able to personalize different parts of the apparel, the format for the 3D model was a challenge but finally, gLTF format emerged the winner for both the web and spark ar experiences
- The size of the 3D models was a challenge. We wanted to use as many polygons as we could for detail but we had to scale down. So please don't mind the quality. IGs file size requirements were lower than FBs
- We felt like the documentation for the native UI module scripting is not so straight forward. You got to have a keen eye to notice textures need to be uncompressed and assigning visibility had to read more on ReactiveMOdule to make things work
- The textures needed dimensions that were in the power of two. Ended up using 512 by 512 though not all textures are able to be tiled seamlessly.
- It was difficult to capture material change action in the video when texture was selected in picker so we just took screenshots.
- Many people in Nairobi use phones without gyroscopes so it was difficult to set up a test group for feedback. As UXers, we believe in the feedback loop.
- The network module was not working so we couldn't communicate with our APIs. We wanted to make one flter that downloads 3D models and materials dynamically based on which product the user interacts with on our e-commerce site.
Accomplishments that I'm proud of
We had to devise our own process flow for making the experience work for desktop users using QR codes. This way we are able to allow users to continue the experience on phones through our custom QR code scanner page. This is a real demonstration that Klurdy is a design studio with UX engineering capabilities.
What I learned
We learned how to take fashion design patterns and create 3D models by ourselves using CLO software. This was a fun experience and we are starting to feel comfortable in 3D garment design. We also got to have a deep dive into Spark AR, gyroscopes, and a different style of reactive programming (We're only used to Rx). We also learned how to use illustrator to make seamless patterns for African prints.
What's next for Klurdy
- We want to port the web app to mobile platforms (iOS and Android) using flutter and desktop platforms (Microsoft and Apple app stores) using electron.
- We are experimenting with Draco, a 3D compression library so that we are able to provide amazing experiences to our users via faster download times. However, we are not sure if we'll be able to run the decoder on the AR effect as a package.
- We are moving forward to look for 50 fashion brands in Europe who will work with us in our beta program as we build the next set of features, like our own AR viewport that has full-body tracking and trying clothing virtually in realtime. It will allow sharing generated video clips that can be shared on social media.
- We also want to explore selling clothes using VR by building a 3D shopping mall that hosts fashion brands. In short, take a deep dive into AR/VR for the fashion industry before everyone realizes the huge opportunity.
- Add avatars to model our clothes, opening possibilities to virtual fashion runways and gamification.
- Add sophisticated material change animations