Quite recently, there were various news reports on the substantial number of fake products being sold on Amazon and multiple e-commerce marketplaces. To add to this, many people have been reporting that their items were ‘fake’ or ‘damaged’ to get a refund on their purchase or try to abuse this system. We wanted to solve this issue by coherently looking at ways to spot fakes in amazon and come up with a new order processing and shipment system that could potentially spot out fakes in third party products, and customers who abuse the current e-commerce system by falsely accusing a product to be defective or damaged.

What it does

Our web app consists of three steps to create a better fake/damaged product moderator. A third-party vendor portal where the vendor must take a picture of the barcode for the product. We use SageMaker’s Barcode reader model to get the UPC for the product. The API then finds the description, price, all the details for the product from various market places. It then asks the vendor for the selling price, if the price is lower than 70% the vendor must explain in detail why he/she is selling this product at such a low price (it’s damaged/refurbished). This to detect if the item is fake. When its time for shipping, a QR code is generated for the vendor to put on the box. This QR code contains the product id which would be used for tracking. The vendor must then upload photos of the product in the box which is then passed through the damage detection model endpoint (Sagemaker). This is the first step to prevent false damage claims by customers. When the item is delivered, the person responsible for the delivery will then log into the web app. He/she will then scan the QR code on the box and take pictures of the product which are then sent through the damage detection algorithm. If some damage is detected the delivery man is asked to check for any noticeable problems and if so a report is filled (database). If the damage isn’t detected the item is delivered to the customer. If a customer complaint is filed about this product through the web app, the customer is asked to fill in the description and specify what he/she thinks is wrong with the product. The customer’s complaint is then compared against the description specified by the third-party vendor using Amazon Comprehend. The first key phrases are extracted from both parties. If similarities are detected then an alert is sent to the customer stating the product description specified what the customer complaining about. If there aren’t many similarities, however, the dynamic text similarity model is used to detect similarities within the text to quantify whether there is a case to be made. If so the customer is connected with a human customer service agent from Amazon to discuss their complaint.

How we built it

The frontend is built with Vue js The backend API is built using Python (Flask) which connects to S3, DynamoDB, Amazon Comprehend, and Sagemaker’s pre-trained models.

Challenges I ran into

With only two people on the team, this project was quite challenging as we had to debug our own solutions and figure out how to use Amazon’s services. One person focused on creating the Frontend and the other focused on the Backend API. However, AWS products were quite simple to use and so we didn’t have any problems making our App. Overall we learned quite a bit from this hackathon about software engineering life cycle, project planning, using Amazon products and understanding trying to focus on what the business requirement when making the App.

Accomplishments that I'm proud of

Our app was a success! We managed to create an MVP with 3 web portals for the vendors, delivery personnel and the customers. In the end, we executed the solution well and we hope if given an opportunity to continue working on this project and find other, more robust ways to “Moderate” the Amazon/e-commerce marketplace.

What's next for The Moderator

We want to create our own classifier and use Amazon Rekognition to find fake photos by third-party vendors to make the product more robust. We would also create a better text extraction classifier to understand and classify user complaints. The more unique the complaints are the harder it is to classify increasing the error rate. We want to create a Convolutional neural network for text classification and using it in conjunction with an image classifier to generate nodes between the too. Ex: Suppose the customer says there was a crack in the vase that he/she ordered, the customer will be asked to take a picture of it and if there is a visible crack, the image is compared with the vendor image that was sent before shipment. This would speed up the process to check for damage/or other noticeable defects.

Share this project: