Cross-Stitch Maker

Inserting an image into the app
Image downloaded to phone

Inspiration

This project was inspired by a personal love of cross-stitching, and a desire to make a cross-stitch pattern generator app that's both ad-free and functional.

What it does

A mobile app for Android that accepts images from the phone gallery, and allows you to produce a low-fidelity cross-stitch pattern using machine learning. The user is allowed to customize the size of their pattern, and the number of different colors of floss. All colors in the pattern map directly to real colors you can purchase at craft stores like Michaels!

How we built it

The frontend was built using React Native managed by Expo. The backend was written in Python, using Scikit-learn's Mini batch K-means algorithm (efficiency is a concern as we wanted to reduce network latency as much as possible). We used Google Cloud Functions to deploy the backend, a serverless API that processes images sent in the request form. Additionally, we used Google Cloud Buckets to store resources needed by the API in a cloud environment (i.e. a dataset containing all colors of cross-stitching floss produced by DMC).

Challenges we ran into

Many of the concepts relevant to this project were new to both of us. Learning how to use React Native as opposed to traditional React was a hurdle. It took many, many hours to understand exactly how user requests should be structured in the front-end, and how responses should be encoded and sent back to the user within the API itself. We had also originally planned to use AWS Lambda for hosting, but we ran into issues with file size limits and had to switch to Google Cloud on the fly.

Accomplishments that we're proud of

The user interface of the application looks really nice to us and exactly fits the aesthetic we were going for from the start. We're also really proud that we were able to set up the API in time to get a working prototype finished.

What we learned

How images are encoded as blobs and base64 strings, and how they're sent in requests and responses.
How to create and host an API
How to pipeline a machine learning model
How to use React Native for UI design