ML products will have a larger impact than ever before as more and more companies incorporate business-critical machine learning products and models into their businesses and production systems to assist in essential workflows. This makes the need to find bugs and fix issues paramount challenges. Additionally, for many of these models, especially deep networks, it's not clear how to formally define or solve these problems. This is why we built a platform that uses the power of crowdsourcing to find and address bugs in machine learning systems.
What it does
Bug bounty platform for finding issues in machine learning systems.
How we built it
We split work between a frontend, backend, and several machine learning APIs that we ran on remote EC2 servers. The frontend was built using react.js and next.js, running as a single page application in the browser that is server-side rendered. The backend was built using bottle.py with sqlite3. It consists of an API that handles user account management, managing the testing suite, and handling remote requests to servers hosting ML models. Additionally, we had a built-in fuzzing module that added random gaussian noise to images to try and break the ML models. This would then email the user when their model was broken, and with the specific input.
Challenges we ran into
There were a lots of moving parts with this architecture, as we not only had the traditional fullstack application, but also had to outsource computation to external services. Additionally, because none of us were designers or UX experts, figuring out how to implement a clean and consistent frontend and user experience was a challenge.
Accomplishments that we're proud of
We're proud that we build a real and functioning product within the span of 24 hours that works for image classification tasks. We resolved many backend issues including authentication, server coordination, fuzzing and image transformation, as well as efficient storage and ranking algorithms for search and leaderboards.
What we learned
Initially, we started out with a project that could automate software patches using AI. We learned that automated software patch generation is a lot more challenging than initially suspected (as in, 5 years and a team of PHD's). We learned to iterate quickly, and decided to pivot to a marketplace for machine learning security. This is a far more tractable problem, and with no less impact.
What's next for Patchwork
We are looking to expand our testing suite to validate ML models designed for different product classes such as image recognition, text processing, and audio. We want to find startups with ML products that would be willing to pay out for bugs found and hackers who want to play with ML products.