Inspiration
Problems from a real story
It has been started from personal experience. Florian, one of our team members from France couldn't live up to his mother's expectation that she asked him to buy Korean beauty products while he was visiting Korea.
A lot of foreigners come to Korea and take a visit to cosmetic road shops with great expectations toward Korean beauty. However, most of them end up leaving the store with light-handed due to the language barrier. Moreover, too many products make them hard to decide which to buy. So they put things that are commonly known as "must-have" items when visiting Korea and leave the other great products they'd fall in love with behind. Is there any way we can help these people? How could we let the world know the real power of Korean beauty?
Start to the dream project
We know there are already many solutions provided by mobile applications but the fact is, it's quite powerless at the front of the Korean cosmetic industries. There is a high level of demands and needs asking to solve this problem. And this service is not just about helping foreign visitors. By providing practical and helpful information on products, this service has tremendous possibilities considering that around 20 million foreigners have a visit to Korea each year. We are to present the ultimate solution that will give big smiles on both the visitors and the industry.
What it does
Definition
The iOS app service that provides a new shopping experience helping to buy unfamiliar Korean beauty products by AR scanner and AI technology.
Pain points
Buying new beauty products when visiting a new area is challenging because of "Hamlet Syndrome" in the cosmetic shop.
We'd like to feel insecure when we aren't able to get ample information on the internet even it's an item catching our eyes. Because of the language barrier, they have no clue to get information on the back or even from the store workers.
Searching each item on every different link, which requires you to type something on it, is inconvenient. And just imagine what it'd be like if you have no idea of how to read the hangul!
Solution
We are going to develop AR live mode search feature that provides them the practical information of the product they need to know before the purchase. So the users don't even need to take a picture for image search on google nor struggle to translate the language. With the optimized services powered by AI technology, they can get the essential information such as what cosmetic ingredients they should take care of. All these solutions will bring the users to end up having a satisfactory experience in the easiest and the most useful way.
How we built it
We have implemented the optical character recognition, known as OCR for recognizing the products. By applying tessearact-ocr and adjust it to cosmetic data, we've improved the model to get better performance. It is also possible, in the application, to reconnect an item by his barcode. Each item attach an unique identifier that is read by the camera and allows us to perform the product search. Augmented reality, via the Apple framework ARKit, is used in order to perform an efficient surface detection increasing the visibiliy of the object we want to analyse. Moreover, we used Papago AI to translate all the information of a given product into the users' local language. This tackles the painful language difficulties through translating the nouns, onomatopoeic or mimetic words that are difficult to be translated into other languages.
Challenges we ran into
We were challenged to train the model to get the desired result and apply customized Papago API. This was to remove the language barrier by removing the need to search for product information by themselves and finding an appropriate noun, onomatopoeic or mimetic words that are impossible to understand for foreigners. Plus, AR live mode that is incredibly convenient to use. We dare think we will be the only team who succeed in the whole that development in all kind of hackathons in the world if we make it.
Accomplishments that we're proud of
Our ultime goal was to provide useful information about cosmetic product to the public. We were able to accomplish that by using the powerful Papago NMP API to translate our product database to every language supported by the platform. For the usability, we decided to try AR live mode that wasn't the original plan due to lack of experiences and time, but we have made it with a beautiful UI/UX in case It even provides a barcode search. On top of all that, We have built our high-tech searching engine that brings the perfect result with the most optimized customized papago API as we planned to in 2 days!
What we learned
We have overstepped limitation of ourselves during this hackathon. The beginning was a simple image search app that uses google API. Developing AR kit or a machine learning model was only theory for future plan. The searched result had also planned using google web search. In fact, what we planned was a simple prototype that we could safely make it in 48 hours. Of course, we could've satisfy with the outcome itself. While leveraging the Naver AI platform, however, we started to think we can put our foot furthur and enjoy the real sprit of a Hackathon. We even didn't sure to finish realizing all the things in time for the submission. However we dared to challenge it. The limitation is made by what we imagine, which means we can also step over by what you imagine. During the three days of the hard core, we encouraged each other. We helped each other. and we grew altogther. And we could see what we've made will help millions of people to meet Korea more, experience Korea more and love Korea more. The times we pour our heat-hearted passion will make a better world, which has started from the spirit of the challenge, the start-up spirit!
What's next for Skana
We have also found a lot of improvements in order to make it for a real service, but we sure how far this service can go. Even though what we made in junction Seoul is a tremendous achievement, we firmly believe it's a baby step of the AI platform's power. For example, the data that will be collected from users has so much potentials to bring a huge revenue to Korean beauty industry. The feedback from the image search can be feeded back to the ocr model for training. Also the user pattern data can provide the customed recommendation for the users from all countries. So that we can develop a solid business model with our own big data. We have already found that it can be applied to the Japanese beauty market as well. For sure, it will change the whole shopping experience all over the world.
Log in or sign up for Devpost to join the conversation.