SMART (Scan, Match, And Retail Transaction)

Inspiration

My inspiration came from observing the daily challenges people face when shopping, both online and in physical stores.

In brick-and-mortar stores, customers often find it challenging to get detailed information about products instantly. They have to rely on store employees for product information, comparison, and sometimes, even for finding the product's location in the store. People often see something they like in the real world, but don't know where to buy it or how to find it online.

In the online world, the problem is the overload of information. Searching for a product often results in too many options, and it becomes time-consuming to sift through all the information and make an informed decision. Not to mention the uncertainty of whether the product will fit their needs in real life, particularly in the case of items like furniture.

I wanted to create a solution that combines the best of both worlds - the tangibility of physical shopping and the information accessibility of online shopping. The inspiration was to empower customers with the ability to instantly access product information, compare prices, check for discounts, visualize products in their space using AR, and complete a secure transaction all in one app.

The idea was to build a bridge between the physical and digital world, offering an innovative, intuitive, and convenient shopping experience. By harnessing the power of advanced technologies like Google Lens, AR, and Square payment processing, SMART was born.

Ultimately, the inspiration behind SMART was to transform the way people shop and make their lives easier, smarter, and more efficient.

What it does

SMART (Scan, Match, And Retail Transaction) is an innovative mobile application that leverages the power of Google Lens, augmented reality, and Square payment processing to provide users with a unique shopping experience.

The main service that SMART provides is the ability for users to scan items using their smartphone's camera and receive instant product recommendations and purchase options. By using Google Lens, SMART can recognize and identify items in the real world and provide users with a seamless and intuitive way to find and purchase products.

Once a user has scanned an item, SMART will analyze it and present them with a range of options for purchasing the product. This includes information on where the item is available for purchase, pricing information, and any relevant promotions or discounts. Users can then complete the transaction using Square payment processing, which is known for its speed, security, and reliability.

How we built it

We use search engine regularly. When we have queries, we can use the search engine like Google to retrieve the most relevant answer. Most of the queries format is text-based. But not most of the time, the text is quite useful to find relevant answers. For example, you want to search for a product on the internet, in this case, a t-shirt, but you don’t know the name of it. How could you find them? Well, you can write the description of that shirt. The problem for using descriptions is that you will get wide varieties of products. And what makes it worse, they will be not similar with the product that you want to search, so you need a better way to retrieve them. To solve it, we can use the image of the product, extracts its features, and use those features to retrieve similar products. We call this concept as content based image retrieval.

Challenges we ran into

Initially our plan was to integrate google vision product search api, So we started with vision and then we faced lots of challenges like building the product set in cloud bucket, indexing the products on cloud bucket, setting up cloud apis in android and backend sides.

At the end getting proper visullay matched results was challenge. Vision result was random.

So finally we decided to make our content based image search engine. For that we have used python.

Accomplishments that we're proud of

Getting the system to recognize an image and matching it to the database then linking to Square terminal for payment.

What we learned

Google Lens

What's next for SMART (Scan, Match, And Retail Transaction)

To hopefully pilot the application with a vendor for real world testing.

Built With

• firebase
• google-lens
• javascript
• kotlin
• ml
• php
• python
• sdk