GitHub repo link is available in the Additional Info section ✌️
Inspiration
As web technology rapidly advances, we are now entering the era of web3. Despite this progress, many users remain more comfortable with web2 due to the lack of user-friendly and meaningful applications in the Web3 space. Most web3 projects have focused primarily on NFTs—minting and staking—which do not offer practical benefits to everyday users who rely on functional Web2 apps.
This narrow focus has resulted in only about 1% of users transitioning from web2 to web3, reinforcing the belief that Web3 applications are not useful. Our project seeks to change this perception by offering a comprehensive solution: a website for sellers and an extension for buyers. By addressing real-world needs, we aim to showcase the practical value of web3 and encourage wider adoption.
Creating a web3 application for shopping is particularly beneficial because trade is a timeless human activity. As long as humanity exists, buying and selling will remain essential. This makes our app not only relevant but also essential in demonstrating the lasting value and practicality of Web3 technology.
Note: As our project video was getting lengthy, we didn't mention the WhatsApp bot & Theta Card for buyers. Please read about these pivotal components in the documentation later. There are many features that we could not show due to the timeframe of the demo video, so go through all of the images that we uploaded in the above carousel, and also read this entire documentation to completely understand our project.
What it does
Our project is made for the buyers, and sellers (brands). The sellers will be using the website part of the project to do their stuff, whereas buyers will majorly use the extension to get things done.
Before moving forward, let's understand why we ask for the owners' private keys during registration
It's easy for someone to provide another person's public wallet address from a transaction explorer, making it difficult to verify true ownership. Therefore, we decided to use a more authentic method to validate the wallet for a user. Theta wallets can be authenticated using keystore files, mnemonics, or private keys. Since we use DESCOPE for authentication (onboarding/logging in entities), we couldn't handle keystore files due to their lack of support for uploaded files in their form flows. Thus, we had to choose between private keys or mnemonics. We opted for private keys as they are shorter and more manageable than mnemonics.
Once the wallet is verified, the private key is immediately deleted. It is never stored, does not go to the server, and is handled directly in the UI. Users need to input the key only once during sign-up. Afterwards, they can log in with their credentials. This wasn't the only reason. Most onboarding users are non-Theta users, so we offer them the option to create a wallet and even top it up with 5 TFuel. This allows new users to join our project and the Theta network through our application.
Website, for sellers/brands:
Brands must complete the Descope authentication process to register on the website. Once registered, they gain access to the products and analytics pages.
On the products page, they can manage their inventory. When adding a new product, they utilize two AI/ML models from Theta Edge Cloud: the custom Stable Diffusion XL turbo model and the CLIP (Contrastive Language-Image Pre-Training) model for zero-shot image classification. A reference image is necessary for creating a product.
Brands can upload their product image or use the Stable Diffusion model, exposed as a server with Gradio, to generate one by providing a prompt. The Stable Diffusion model always returns a new image because its parameters are randomized each time. In either case, the image is processed by the CLIP model, which identifies the main object and returns a label. This ensures consistent labelling for products uploaded by different brands.
On the analytics page, brands can view data distribution to understand their rival brands registered on the application and analyze buyers' demographics. They can also initiate targeted email campaigns by filtering their target audience and sending detailed emails about upcoming sales or campaigns. Additionally, they can attach a brochure to enhance the email's impact.
Extension for buyers
We made an extension for the buyers because it takes up the minimum hardware space and pixel space on the user's divide, and still delivers the same impact as a website.
When creating an account on the extension, you must provide your phone number, including your country code, so Twilio can send you a verification message. Ensure you do not click outside the extension during registration, as this will close the extension and disrupt the process.
After logging in, you must enter your password each time you open the extension. This security measure protects your Theta account and the valuable Tfuels it contains. Once you provide the password, you can access and use the extension.
On the recommendations page, you'll see various products that match your interests and the information you provided when creating your account. The brands section displays products from all brands, not just those matching your interests. In this section, you can access three key features: a sorting/filtering tool to refine your search, a Text to SQL AI tool that fetches data based on text input commands, and an CLIP-ML model file upload feature that activates when you upload an image.
In the demo video, we didn't show how the ML model in the extension works. Let's see how it works here! We have provided a screenshot of this in the image carousel above. After you upload an image to the input field, the model extracts the top 5 labels present in the image. It then fetches objects that match any of these labels. For example, if you upload an image of your dream room with various objects like a cup, table, phone, bed, chair, shirt, and more, the model will identify the 5 most recognizable items and fetch similar objects. These will be displayed on the UI, allowing the buyer to see all relevant items in a single block without needing to filter or manually search.
In the feedback section, one can use the Gen AI model to generate text by just giving out a prompt. We have added the screenshots for the same in the image carousel. You have to type in your prompt between "* *" and then the model will return with the tailor-made content (max length of 500 characters).
In the rewards section, users can unlock rewards by sending their first feedback and logging into the extension for 5 consecutive days. For providing feedback, users receive 10 Tfuels, and for logging in continuously for 5 days, they receive 20 Tfuels.
Buyers can add items to the cart by tapping the plus icon. Orders should be completed on a webpage to avoid accidental cancellation within the extension. After adding all items, click the cart icon at the top to review your order. Then click on the Your cart text to navigate to https://versatilevats.com/theta2024/checkout.html. Finally, open the extension again, navigate to the cart, and click Checkout to complete your order.
For demo purposes, 1 Tfuel is equivalent to $10, allowing users to execute orders with fewer Tfuels. On the checkout page, you can see all four NFTs available. Each NFT is allocated to the buyer after reaching certain milestones and comes with specific perks or discounts, which are listed with each NFT. One can get a maximum discount of 15% once the last NFT is unlocked.
Past orders can also be viewed on the checkout page. They include all major details such as the items purchased, the brand, total price, discount, and more.
How are Tfuels sent to the brands? Assuming an order total of $170, which equates to 17 Tfuels (with a 0% discount for the first order), here's the process:
- The entire 17 Tfuels are deducted from the buyer's wallet and credited to the admin account.
- In the backend, the admin account sends each brand its share of Tfuels in separate transactions.
- For example, if the buyer purchased items from Nestle, Adidas, and Ikea, the admin will distribute the Tfuels accordingly.
- This process ensures the user experiences zero delay, while the backend handles the individual transactions to the brands.
WhatsApp bot for buyers The extension can only be opened on laptops/computers. When away from these devices, buyers can use the WhatsApp bot to access data from the project. This bot allows users to view products and brands listed on the application in a read-only manner; no changes can be made via the bot. Screenshots in the image carousel demonstrate how the bot works.
We couldn't provide a link to use the bot because it was created and tested using Meta's developer account, restricted to our numbers. Making it live requires a hefty fee, so we opted to develop it locally and provide screenshots as proof of its functionality.
Theta Card for buyers Certain items in the application are available for pickup from a physical store. To apply for this, the user must submit an extra form. Once approved by the admin, the user receives a real NFC card, which is loaded with Tfuels after placing an order that includes a pickup item. When the card has a balance, the user can visit the physical store, and use the card to collect the order, and the required Tfuels will be deducted from the card, providing a seamless experience for the buyer.
How we built it
We built the user interface for the website and the extension using Bootstrap and HTML. JavaScript was utilized to provide functionality to the DOM elements. Node.js was used to spin up the servers. The CoolNFT smart contract was used to mint NFTs and transfer them.
Apart from that, we used the Theta JS NODE SDK to integrate our project with Theta. We configured it in a node js server to expose the various features as endpoints and we did the following things in it:
| Endpoint | Description |
|---|---|
| /getWallet | Validates the Theta wallet (with the help of the private key, mnemonic, or keystore) |
| /createWallet | Creates a new wallet |
| /sendTfuel | Sends Tfuel to the desired account |
| /deployContract | When a new buyer signs up on the extension, the smart contract mints a new set of 4 NFTs that will be provided to the buyer upon unlocking. At the time of unlocking, they are transferred from the admin's account to the buyer's account. |
| /transferNFT | Transfers the NFT from one wallet to another |
| /sendEmail | Sends an intimation email to the brand/buyer whenever they opt for a new Theta account during registration |
As we have used the already-made Stable Diffusion Model for image creation, we had to set up a Gradio Server on our own to integrate the model with our application. Strength, guidance, steps, and seed are some of the parameters on which the Stable Diffusion model depends. So, whenever a request comes in to get an image for a prompt, the model fetches unique values for each parameter, ensuring that a new image is generated each time (which is cool 😉).
The Clip ML model was developed by our team and is one of the more complex models to use. After tailoring it to our needs, we exposed it via a Python Flask server.
Apart from these, there are a ton of files that went into making the project live and you can see those in the GitHub repo.
Challenges we ran into
From the start, we faced many challenges due to the diversity included in the project. Developing a website is completely different from developing an extension or a WhatsApp bot. Creating a hardware setup was also challenging, as it required substantial hardware knowledge, and we spent quite a bit of time understanding it. Vishal, from our team, was super-active in the Discord channel and ensured that we fully utilized the Theta Edge Cloud services. Getting the first custom template running was a breakthrough, as it paved the way for using the Edge Cloud to its full extent. Some of the notable challenges that we faced are:
1️⃣ Integrating the Theta Node.js SDK with the Descope authentication model was a bit daunting, as we had to re-run the flow repeatedly to ensure the correct API calls were made and their responses were handled properly.
2️⃣ Making the Gradio server for the Stable Diffusion model was confusing because, while we could easily use the model from the Theta Edge Cloud website, we didn't initially know how to expose the model using Gradio. We aren't just generating an image from the prompt; we are also saving it on our server because Gradio returns a temporary link that gets deactivated after several hours. So, not only did we fetch the image from the prompt, but we also saved it onto our server.
3️⃣ Setting up the CLIP zero-shot image classification as a Flask app was challenging due to its many dependencies. We had to choose the lightest ones to prevent the model size from exponentially increasing.
4️⃣ Apart from these, we had a hard time completing the ordering flow for buyers via the extension. We had to devise a smooth way for data to be shared between the extension and the checkout page, and vice versa.
Accomplishments that we're proud of
Entirely hosting our application on Theta Edge Cloud is a great opportunity for us, as we created five custom templates on Theta Edge Cloud and only used one standard/pre-made template from the models library. So, let's review the template deployments that we did to make our project live:
- Image Generation Model (ID: stablediff6cq10hx8b7): Running on a GT1 instance
- Gradio Server (ID: gradioservs5aprev8oa): Running the Gradio server that returns the URL of the image generated from the prompt. Running on a C1 instance
- CLIP ML Model for image labelling (ID: imagerecogpp6ekvqkn7): Python flask app that is running the CLIP model. Running on a C1 instance
- Node JS Server (ID: nodejsserv5ew42q7of8): The basic node server that handles each backend request by exposing the services as endpoints. Running on a C1 instance.
- Theta JS Server (ID: thetanodejbav3zvutrb): Handling all of the Theta backend calls and making this project possible. Running on a C1 instance
- Theta BASH UI (ID: thetabashuzgz3haaf91): Hosting the website for sellers/brands. Running on a C2 instance
Out of six deployments, three are for AI/ML models, two are for Node servers, and one is for the UI. Considering we knew nothing about Docker before this event, containerizing and deploying each component of the project on Docker to run them on Theta Edge Cloud is a great achievement, and we are thrilled with the results.
Apart from these, we used a couple of AI models in the extension: one for generating text, and while other to convert text to meaningful queries to fetch data from the backend.
These accomplishments validate our project as one of the best submitted in this event, qualifying us for both the Theta Edge Cloud and Generate AI tracks (but we are still aiming for the top spot 😉).
What We Learned in This Event
During this event, we learned a multitude of skills and gained valuable experience. Here are some key takeaways:
Theta Edge Cloud Utilization: We discovered how to effectively use Theta Edge Cloud services, creating five custom templates and leveraging the platform's capabilities to the fullest extent.
Gradio Integration: We learned how to set up a Gradio server for our Stable Diffusion model, enabling us to generate and save images from prompts, despite initial confusion about exposing the model via Gradio.
Handling Complex Dependencies: Setting up the CLIP zero-shot image classification model as a Flask app taught us how to manage complex dependencies and optimize for model size.
Authentication and API Integration: Integrating the Theta Node.js SDK with the Descope authentication model required us to repeatedly run flows to ensure correct API calls and response handling, deepening our understanding of authentication processes.
Order Flow Optimization: We devised a smooth method for data sharing between the extension and the checkout page, ensuring a seamless ordering flow for buyers.
Docker and Deployment: We went from knowing nothing about Docker to containerizing and deploying all components of our project, enabling us to run them efficiently on Theta Edge Cloud.
Overall, this event provided us with a comprehensive learning experience, enhancing our technical skills and project management capabilities.
Do check out GitHub to see the credentials that you can use to access the extension. For the website, you have to create your own account as it is tied with each email and the email is required to access an account on the website
Log in or sign up for Devpost to join the conversation.