Federated jaw lesion classification on Ethereum

A web application for submitting AI models trained on local radiology images to classify jaw lesions. Oracles through ChainLink are used to aggregate the local model to a global model.

Comment

Inspiration

I was inspired by the need for unbiased privacy preserving machine learning models from all over the world for lesion classification. I saw some projects using federated machine learning and thought it would be beneficial for it to be on a decentralized platform, where it can leverage the financial management and model ownership management using the immutability of blockchain technology for the traceability of the local models, so there is no question about ownership, reliability, and credibility.

What it does

Web application that connects to metamask and stores the user's address to a sqlite database on initial registration. It displays the user's wallet account balance and retrieves the price feed from chainlink oracle to give the balance in USD on the landing page. User can upload an AI model file (.h5 or .json) to the application to be submitted to the smart contract on the kovan testnet (contract address: "0xCEbd5b8a5128c3B566f64415911C10a940ba844e") after they have trained the model locally on their radiology databset. Although the application allows for uploading AI model files, it does not yet send the whole model to the smart contract. The application is currently hardcoded to send a string to the smart contract to be stored for later retrieval when the user submits an xray file for prediction. The application also saves each transaction sent to the smart contract in the sqlite database and displays all the user's transactions on the history page.

How we built it

I built a web application using flask and python and installed web3, metamask, and infura provider on the python backend to interact with the smart contract that I deployed to the kovan testnet . Currently, the application retrieves from the smart contract a string to display on the flask frontend and sends a transaction to the smart contract to store a new string value. The flask front end also connects to metamask and stores the address into a sqlite database when the user registers. The smart contract is built using vscode IDE and brownie. Web3 is used within python to get the price feed data from a chainlink oracle through the smart contract.

Challenges we ran into

Challenges I ran into are related to understanding how federated machine learning works and sending model files to a smart contract or chainlink oracle. This challenge is something I'm still researching and have not solved yet at the time of this submission.

Accomplishments that we're proud of

Learning how to write a smart contract in solidity using brownie and deploying it to the kovan testnet. Then using web3 and metamask within the flask/python application to interact with the smart contract, both through a retrieval call and submitting a transaction to store data on the contract.

What we learned

-How to create a smart contract and deployed it to the kovan testnet. -Use web3 to retrieve from the smart contract and send transactions to it with python. -Connect to metamask from javascript used in the web application. -learned about nextjs and modal through the chainlink video tutorials by Patrick.

What's next for federated health jaw lesion classification on Ethereum

Learn more about Chainlink External Adapter framework to see if it can be used and convert the web application to use nextjs and modal possibly.

Built With

Share this project:

Updates