GlioAI: Automatic Brain Tumor Detection System

Automatic Brain Tumor Detection Using 2D Deep Convolutional Neural Network for Diffusion-Weighted MRI


Part I: Summary

Part II: Results

Part III: Conclusion and Future Work

Additional Documentation


GlioAI is an automatic brain cancer detection system that detects tumors in Head MRI scans.


  • Primary malignant brain tumors are the most deadly forms of cancer, partially due to the dismal prognosis, but also because of the direct consequences on decreased cognitive function and poor quality of life.

  • The issue with building an effective machine learning system for a medical imaging-based task before was that there was not enough data. Using transfer learning and implementing slight architecture modifications to adapt the model to our dataset allowed GlioAI to perform at over 95% accuracy.

  • Given the context of a transfer learning approach to a feature detection problem, it is crucial to safeguard the probability of model overfitting with prevention methods involving data augmentation, applying data normalization and dropout layer.

  • Magnetic Resonance Imaging is a new method that has emerged for improving safety in acquiring patient image data. The utilization of these imaging tools are not yet fully maximized due to the variable of human operation within detecting cancers without enough time to make an accurate prognosis.

  • Because manual image feature extraction methods are very time inefficient, limited to operator experience, and are prone to human error, a reliable and fully automatic classification method using MRI data is necessary for efficient cancer detection.

  • To begin solving this issue, we propose a fully automatic method for brain tumor classification, which is developed using OxfordNet, a convolutional neural network that has been trained on over a million images from the ImageNet database.

  • We can further enhance the usability of this tumor detection system by building a web application that stores the trained model in the back-end.


Reduce Mortality Rates

  • Create a model that will remove the variable of prognostic human error to improve patient survivability.

Controlling Treatment Output

  • Control the outcome in order to build a system that will mitigate human error and mortality rates.


  • Accelerate the process of deployment for deep-learning based application.


  • Build a cost effective solution that reduces treatment costs via automation.

Usability + Accessibility

  • Create a user-friendly web app that will allow physicians and patients to easily upload their MRI data and receiving data reports and diagnostic results.

Application Workflow

  • The user will import an image of a brain scan and the image will be sent as an http request through the resource server (REST API)
  • The API will deliver the request to the back-end model
  • The model will return a diagnosis back through the REST API server and to the front-end UI for the user.

Back-End Design: Implement Convolutional Neural Network

We will be using a deep convolutional neural network, which is a neural network with a set of layers that will perform convolutions, pooling the set of regions of the image to extract features, along with with a softmax function that translates the last layer into a probability distribution.

Training Method

  • We are only interested in applying transfer learning, which relies on training based on previously learned knowledge with a corpus of different large-scale datasets.

  • Because we are given a low volume of training data and are working with images, we decided to use VGG16, a state-of-the-art convolutional neural network with 16 layers to increase the probability of attaining a greater model accuracy.


The Brain MRI Images for Brain Tumor Detection was used to train the model which had 253 brain MRI scans.

The model was trained on 239 images belonging to two classes and tested on 14.

Experiment and Results

  • I: Model and Training
  • II: Comparative Model Results
  • III: Data Evaluation

Model and Training

The model consists of:

  • CNN Layer
  • Average Pooling Layer
  • Dense Layer
  • Fully Connected Layer
  • Loss Function: Categorical Cross-Entropy
  • Optimization Algorithm: Adam

The model is trained on 25 epochs.

Comparison of Model Performance

Transfer Learning

  • Transfer Learning Accuracy: 97%
  • Transfer Learning Loss: 13%

No Transfer Learning

  • No Transfer Learning Accuracy: 76%
  • No Transfer Learning: 49%


When comparing the results of the different models that were trained, it is clear that the transfer-learning based model is the most accurate deep learning model to deploy for the web app.


  • Given that we can precisely automate the process of detecting whether a brain tumor is present in a patient or not, while simultaneously accompanying it with an easy-to-use user interface (for the doctor + patient), hospitals and patients will be able to simplify their workflow for detecting anomalies much earlier and are able to capture it with precision without having to sacrifice accuracy.

  • To further add, healthcare providers will be able to adjacently use applications that are built on top of the rapidly evolving tech infrastructure for care delivery with less friction of accessibility and utilization (via web).

  • There are many improvements to make within the models themselves to account for more diverse and unpredictable anomalies, which can be effectively improved in a cost-effective manner via generating more patient data to train the model using GANs.

  • After further model retuning and additional training optimization, GlioAI can specifically meet the pain points located within diagnosing brain tumors from MRI head scans, for brain cancer specialists and brain oncologists alike. Heading to a future where knowledge is aggregated and integrated with automated cancer detection systems in order to cut down diagnosis time over 1000-fold, from around 14 days of full reports to nearly 10-15 minutes, given the infrastructure for the crowdsourcing platform is built and incentive structures (via gig-based crypto token) and are aligned with verified physician users

  • In this coming decade (2020-2029), the necessity for automation within care delivery will hopefully be deployed at scale, putting the core central focus of the patient back into the hands of the care providers, while lining up monetary incentives for all parties involved via an inverse system between efficiency and cost with automation.


  • I: App
  • II: Neural Network Architecture
  • III: Web Platform Engineering
  • IV: Reflection


  • Add sign-up page for users
  • build API so medical developers can integrate the prognosis tool into their applications
  • Add additional action buttons to allow patient to take action on prognosis (via booking appointments, getting directions to local clinical spaces)
  • Build out CRUD properties for user profile and action buttons in terms of adding notes feature on the web page, etc.

Neural Network Architecture

  • Build General Adversarial Network in order to compensate for scaling data augmentation methods to generate diverse sets of medical data to train the model
  • Build feature that outlines the tumor-infected nodules for the radiologist in order to prevent accidental treatment for healthy tissues in the brain

Web Platform Engineering

  • Build out crowdsourcing platform so users (certified doctors who are verified via medical school email) can assist with machine-based diagnostic decisions (crowdsourcing platform for brain cancer detection (initial MVP, scale and branch out to other specialties later)
  • Incentivize (platform) users with app-specific crypto tokens to reward them in proportion to the amount of value they export to assist in helping other physicians with making a prognosis etc.


Given the current state that the model itself has been trained on a limited set(s) of patient MRI images with great accuracy, there is alot of area for improvement in terms of deploying extensive data augmentation (diversity of input image data for training), feature design, and overall application engineering and usability.

Future of GlioAI

  • I: Main Focus for the Future
  • II: Developmental Scope for 2020s
  • III: Reflection on Targets

    Phase I: Integrate Cryptoeconomic Mechanisms within Crowdsourcing Web Platform for Radiologists

  • The future of GlioAI lies in the idea of turning into a decentralized and pseudononymous crowdsourcing platform for medical practicioners and verified physicians and healthcare providers within the context of deep-knowledge tasks to further prune outputs from machines & automated systems ranging in disease detection and other areas in health.

  • Game design mechanisms can be built out within the crowdsourcing platform in order to line up incentives for users to offer verified feedback that gets simultaneously ranked.

  • Propagandistic behaviors cannot occur because of the account verification process in order to create content or rank/upvote other posts (containment + authentication-based friction).

  • Enhancing treatment results via crowdsourcing platform specifically for verified doctors and healthcare providers (verified via school email + State ID)

  • Integrate gig-based cryptoeconomic mechanisms in order to incentivize (digital) teledoctors to be able to easily generate income via telemedicine tasks to ensure accuracy of diagnosis within timely conditions via providing direct emotional support + answer questions and make clarifications.

  • Crowdsourcing platform + machines = data-driven digital healthcare ecosystem

Phase II: Deploy Telemedicine Mechanisms for Digital Doctors for Treating/Diagnosing Brain Cancer

  • We can further deploy use of these systems by integrating drones to ship treatment medicine with tutorials on the web platform (goal is to make treatment methods open source and qualitatively aggregated together by verified doctors, also self-improving mechanism in terms of data and understanding)

Reflection on Scope/Targets for the Future

  • The number of improvements to make are immense, and the roots originate from one single automated brain cancer detection system.
  • We can build out models and deploy GANs for virtually all image-based radiology tasks first and then shift the focus from just digital radiology (platform + machines) to streamline processes for other diseases.


  • Python

Deep Learning

  • Keras
  • Tensorflow
  • Matplotlib
  • NumPy

Web Application

  • Flask
  • CSS
  • Javascript
  • HTML5




Icon by Srinivas Agra from thenounproject


Contributions are always welcome! For bug reports or requests please submit an issue.



Built With

Share this project: