Masterpiece

Masterpiece is a cross platform mobile app that creates an efficient art market through machine learning and social ecommerce.

Comment

Masterpiece

📍 Overview

Masterpiece is a cross platform mobile app for art enthusiasts, artists, gallerias and investors to explore, discover, purchase, and sell artworks. The app aims to bring together artists, galleries, investors, and art enthusiasts in a seamless and modern manner. The "killer app" of Masterpiece is our in-house ai model Patina; Patina is a fine-tuned online ResNet CNN for the purpose of generating the price of a painting based on training data. Patina is used to inform investors who are determining the value of an painting and to inform artists on the value of their painting. The model is retrained whenever a purchase is made, which keeps the model up to date, and, therefore, creates an efficient art market.

💡 Inspiration

In 2005, unbeknownst to the auctioneers, Leonardo Da Vinci's Salvator Mundi was presented at St. Charles Gallery, it was subsequently acquired for $1,175. Fast forward to 2017 and it was sold at Christie's for a staggering $450,312,500. Disparities like these often occur in the art industry partly because it is difficult to determine the value of a painting. Masterpiece is an attempt to bridge this gap.

📚 Features

Masterpiece consists of users. A user is anyone who has created an account. Once an account is created a user can search for a specific user, view their homepage, or view and customize their profile. Any user can upload a painting with its name, a short caption, and the price.

🏗️ Architecture

The frontend was built using React Native, Expo, and Typescript. We choose React Native because we wanted to build this app once and have it shipped to android and ios platforms. The frontend was built with bleeding-edge Expo libraries for routing and device driver access, making our app scalable and future proof.

In the backend we used Typescript, Express.js, PostgreSQL, Redis, Python, Flask, Keras, Tensorflow, Pillow, EC2, and S3. The backend is broken down into the "social backend" and the "machine learning backend". The social backend handles the interaction between the frontend and the backend and the machine learning backend handles the interaction between the backend and Patina. The entire backend is hosted in an EC2 instance and all the images are stored in an S3 bucket.

🥊 Challenges

Going in we had solid full-stack development skills, but nothing extensive enough to build an end-to-end platform like Masterpiece. Here were the challenges:

  • Building the React Native UI: React Native(specifically expo) has changed a lot since we've used it and most of the tutorials are based on the older versions. We had to rely almost completely on documentation and faith.

  • Designing the APIs: Implementing the APIs was relatively simple, just follow the API design. Designing the API on the other hand was a challenge; we had to coordinate with frontend and the database to provide the necessary information that was also available.

  • Uploading images to S3: This was the first time any of us handled uploading images, so it was probably the feature that took the longest. We had to learn about FormData and sending base64 encoded images.

  • Training Patina: Since we took an off-the-shelf model and fine-tuned it, we didn't have to worry about the technical aspects of building a CNN, but because we had our own model, we had to collect data, do feature engineering and train the model locally. That was a challenge.

🎉 Accomplishments

We were able to build an end-to-end full-stack cross-platform cloud-first mobile application that relied on an online CNN model and connect all the components seamlessly. The APIs were well designed and our monorepo is well documented.

🤓 What we learned

We learned:

  • Project management
  • Modern React Native (Expo)
  • Machine Learning
  • Teamwork
  • S3
  • EC2

🔮 What's next for Masterpiece

We unfortunately didn't have to time to fully integrate the ecommerce portion of the app. That's the next priority. We also have a ton of data that isn't being use or at least not being used to its possible extent, so we plan on adding more features with that data; for example paintings have names, captions, and likes, but at this this they are never used.

Share this project:

Updates