Inspiration
As students whose current interest are recommendation models, we saw this hackathon as an exciting opportunity to apply our skills to a real-world challenge and gain hands-on experience with the relevant technologies. Moreover, the chance to contribute to TikTok's algorithms was a major motivator.
What it does
Our solution to expand the TikTok shop more widely across the platform is by incorporating hyperpersonalisation by extracting user interaction data such as product clicks and user searches. This interaction data will then be placed into our model, which then makes recommendations in a card that will be displayed when the user scrolls in the For You page.
Extracting User Circumstance Data
The user circumstances will be extracted from the search bar of the main For You page. Why? Sometimes, the FYP does not necessarily have content or video that we might be interested in. So what do we do? We query the search bar to see relevant videos. With this - we saw an opportunity to extract user circumstances from the search field. In addition to the frequent use of the FYP, FYP also give users the flexibility of typing whatever they want into the search bar - enabling for greater personalisation.
Integration of the TikTok Shop into the "For You" page
Based on the extracted user circumstances and user-product interactions, the ShopFYPCard will be displayed to the user when he/she scrolls through the For You page. The products shown in the card are personalised to the user. The user is also given the option to click on the product and be redirected to the product in the TikTok shop.
How we built it
The system is comprised of five main components: The backend, implemented with a lightweight Python-based web library called Flask the product and user circumstance encoder, implemented using BERT language model Vector Search with FAISS for similarity search the frontend/user interface written in React Firebase, which is where product and user information are stored.
Frontend
React was used as the frontend for similating the TikTok For You Page, Search Page, and Product Page.
Backend
The backend was implemented using Flask, a lightweight Python-based framework for developing web-based applications or APIs. For our system, the backend handles several services which can be accessed through endpoints.
Encoding Model
The team has utilised the BERT (Bidirectional Encoder Representations from Transformers) language model to encode user searches (circumstances) and product-based information. The rationale on utilising this model is due to the nature of the problem - which is to understand context of user searches and product information. As such, an encoder model is required. The bonus of the BERT model is its bidirectional attention capability, which means that it is able to understand not only the context given prior tokens, but also subsequent tokens.
Similarity Search Algorithm
The team employed Meta's FAISS to enable more efficient vector lookup for searching and recommending products to the user. Given a search or product vector, the algorithm efficiently computes the L2 (euclidean) distance between the input vector and the vectors stored in the vector table. The products with greatest similarity to the input vectors are the products recommended to the user.
Database
Firebase serves as the database, storing user data (e.g., product interactions, user circumstance searches) and product data (e.g., product embeddings). User data was generated using LLM models to create a realistic dataset for testing and development purposes while Products data were retrieved from the Amazon Products Dataset.
Challenges we ran into
A significant challenge was training our models effectively with the limited resources available. The large dataset we were working with required substantial computational power, and we had to optimize our approach to reduce data volume and speed up training. This took several days.
Accomplishments that we're proud of
Despite the challenges we faced, we are proud to have successfully developed a functional prototype that demonstrates the potential of hyper-personalized product recommendations within the TikTok platform. We're particularly proud of:
- Implementing a real-time recommendation system that responds dynamically to user interactions.
- Leveraging advanced language models and similarity search algorithms to deliver highly relevant product suggestions.
- Developing a search engine model that understands users’ intent.
What we learned
Throughout this hackathon, we gained valuable hands-on experience with many technologies. We learned the importance of efficient data processing and model optimization, especially when dealing with large datasets. We also deepened our understanding of user behavior and the nuances of creating personalized experiences on social media platforms.
What's next for us
We plan to fine-tune the BERT model with more human-like user search data to improve the user-product recommendation accuracy. Our goal is to enhance the similarity search so it can accurately interpret user inputs like "I want to buy shoes for my 10-year-old son who likes Ben 10" and accurately return the most relevant products. This involves:
- Training the model with possible user queries
- Refining product embeddings to capture user preferences and context better. These improvements will help provide more accurate and relevant product recommendations. Overall, we are excited to continue iterating on our solution and pushing the boundaries of personalized shopping on TikTok.
Log in or sign up for Devpost to join the conversation.