LungScan AI

The lung cancer detection using the power of AI.

Comment

Inspiration

Lung cancer remains a leading cause of cancer-related deaths worldwide. Early and accurate diagnosis is paramount for effective treatment and improved patient outcomes. However, current diagnostic pathways can be time-consuming, and human interpretation of complex CT scans can sometimes be subjective. Our inspiration for LungScan AI stemmed from the critical need to leverage the power of Artificial Intelligence to accelerate this process, provide rapid insights, and support clinicians in making more informed, timely decisions, ultimately contributing to saving lives.

What it does

LungScan AI is an intelligent diagnostic assistant designed for rapid analysis of lung imaging scans. It performs a multi-stage classification:

Image Type Validation: Automatically identifies if the uploaded image is a valid CT scan or X-ray, filtering out irrelevant images. 3-Class Lung Condition Classification: For valid scans, our fine-tuned deep learning model precisely classifies the lung condition into one of three categories:

  1. Normal: Indicating no signs of malignancy.
  2. Adenocarcinoma: Identifying the most common subtype of lung cancer. 3.Other Cancer Type: A clinically responsible classification for images detected as cancerous but not fitting the Adenocarcinoma profile, signifying malignancy but requiring further specialized diagnosis.

Patient Data Risk Analysis: Beyond images, LungScan AI processes structured patient data (e.g., age, smoking history, symptoms). Our AI agent analyzes this information to assess potential lung cancer risk factors, providing a holistic view alongside imaging results. AI Safety and Guardrails: A dedicated AI safety agent is built into the system to rigorously evaluate all user prompts and generated content. It ensures that the interactions are appropriate, prevent harmful or irrelevant responses, and maintain a focus on medical and ethical guidelines, protecting against misuse. Comprehensive Report Generation: After all analyses (image, patient data, and safety checks) are complete, LungScan AI dynamically compiles a unified, human-readable medical report. This report synthesizes findings from all relevant sections, providing a clear, actionable summary for clinicians. AI-Powered Explanations: Beyond just a label, LungScan AI integrates a large language model (LLM) to generate clear, concise, human-readable explanations and implications for each diagnosis, enhancing understanding and communication for healthcare professionals.

How we built it

LungScan AI is engineered as a full-stack application, combining robust backend processing with a dynamic and intuitive user interface. • Backend: The core of LungScan AI's intelligence is built with Python (FastAPI), serving as the robust API endpoint framework for seamless data interaction and model orchestration. • Frontend: For the user interface, we developed a responsive web application using React.js, providing an interactive and modern experience for clinicians. • Styling: The frontend's aesthetic and adaptive design were meticulously crafted using Tailwind CSS, ensuring a clean, mobile-first, and highly customizable visual presentation. At the heart of our image analysis lies PyTorch, leveraging a pre-trained ResNet-based deep learning model for lung cancer classification and a separate model for image type detection. Our key development process involved: • Model Fine-tuning: We significantly enhanced the lung cancer classification model by fine-tuning its pre-trained weights. This was achieved by: o Utilizing the dorsar/lung-cancer dataset, which provided specific examples for Adenocarcinoma and Normal lung conditions. o Augmenting this with an additional ~360 mixed cancerous/non-cancerous images (238 cancerous, 126 non-cancerous) to boost robustness. We intelligently mapped the dorsar's Adenocarcinoma and Normal classes, while assigning the general "cancerous" images from our mixed dataset to a new 'Other_Cancer' class. o Employing torch.utils.data.ConcatDataset to seamlessly combine these disparate datasets, creating a much larger and more diverse training pool. o Implementing class weighting in the CrossEntropyLoss function to effectively mitigate dataset imbalances and ensure fair learning across all three classes. • LLM Integration: We integrated Vertex AI's Gemini Flash model for dynamic generation of human-readable explanations, utilizing Function Calling to provide structured diagnostic narratives, patient risk assessments, and comprehensive reports. Also, the google ADK starter pack. • Environment Management: Conda environments were meticulously used to manage complex dependencies, particularly for PyTorch, TensorFlow, and other scientific libraries, ensuring a stable development and deployment environment. • GPU Acceleration: We leveraged an NVIDIA GTX 1650 GPU and a CUDA-enabled PyTorch installation to drastically accelerate model training, reducing epoch times from minutes to seconds and enabling rapid iteration.

Challenges we ran into

Developing LungScan AI presented several significant technical hurdles, which we successfully navigated: • Version Compatibility and Library Conflicts: A major challenge was ensuring seamless version compatibility across various components. We encountered ModuleNotFoundError for TensorFlow due to Python 3.12 incompatibility, requiring a shift to a more stable Conda environment with Python 3.10 and specific CUDA-enabled PyTorch installations. Furthermore, managing the precise versions of libraries within the ADK starter pack environment while integrating external deep learning frameworks (PyTorch, Hugging Face datasets, scikit-learn) demanded careful attention to dependency resolution to prevent conflicts. • PyTorch API Changes: Hit a TypeError for an unexpected verbose argument in ReduceLROnPlateau due to PyTorch version differences, which was resolved by removing the unsupported argument. • Dataset Limitations & Imbalance: The most significant data-related challenge was the inherent limitation of available public datasets. The primary dorsar/lung-cancer dataset, after initial inspection, primarily contained "Adenocarcinoma" and "Normal" samples, lacking specific, robust labels for "Large Cell Carcinoma" or "Squamous Cell Carcinoma." This forced a pragmatic, yet effective, shift from a 4-class to a 3-class classification strategy (Adenocarcinoma, Normal, Other_Cancer), and required careful handling of the mixed binary dataset to correctly augment these new combined classes. • Orchestrating Multiple AI Models: Integrating and managing the workflow between disparate AI models – including a PyTorch-based image type classifier, a PyTorch-based lung cancer classifier, and multiple LLMs (Vertex AI's Gemini Flash for explanations and a separate LLM for report generation) – within a single FastAPI backend added significant complexity. Ensuring smooth data flow, error handling, and effective tool calling between these agents was a considerable challenge.

Accomplishments that we're proud of

Highly Accurate 3-Class Classifier: Achieving an impressive Test Accuracy of 97.13% and a Macro F1-score of 96.30% on our 3-class model. This includes excellent F1-scores for Adenocarcinoma (0.9849), Normal (0.9726), and particularly the newly defined 'Other_Cancer' (0.9315), demonstrating the model's robust ability to differentiate. Intelligent "Other Cancer" Classification: This nuanced category is a major point of pride, allowing the model to indicate malignancy without making an unverified specific subtype prediction. This is a crucial step towards safer and more ethical AI in medicine. Successful Fine-Tuning Pipeline: We built a robust data ingestion and fine-tuning pipeline that effectively leverages both a specialized Hugging Face dataset and supplementary local data to significantly improve model performance. Seamless LLM Integration: The ability to generate contextually relevant, human-readable explanations directly from model predictions greatly enhances the system's utility and user experience. Overcoming Technical Debt: We systematically debugged and resolved numerous complex environment, dependency, and code structuring issues, leading to a stable and functional deep learning pipeline.

What we learned

This project provided invaluable lessons: • The paramount importance of understanding your dataset's actual content and class distribution, especially in medical imaging, rather than relying solely on high-level descriptions. • Deep practical experience in debugging complex Python environments, including venv vs. Conda, init.py for package recognition, and handling DataLoader issues. • Effective strategies for fine-tuning pre-trained models and the critical role of class weighting in addressing data imbalance for multi-class classification. • The power and potential of integrating specialized deep learning models with large language models to create more holistic and user-friendly AI solutions. • The iterative nature of AI development and the importance of adapting strategies based on real-world data constraints

What's next for LungScan AI

What's next for LungScan AI We envision several exciting future developments for LungScan AI, focusing on both enhanced AI capabilities and a robust, user-centric clinical platform: • Expand Specific Cancer Types: Our primary goal is to acquire and integrate more rigorously labeled datasets (e.g., from TCIA with histopathology reports) to enable more granular classification beyond "Other_Cancer," specifically for Large Cell Carcinoma, Squamous Cell Carcinoma, and potentially Small Cell Lung Cancer. This will provide even more precise diagnostic insights. • Prognostic and Risk Assessment: Explore integrating comprehensive clinical patient data (e.g., patient demographics, smoking history, co-morbidities) alongside image analysis to provide a more holistic risk assessment and even prognostic indicators, aiding in personalized treatment planning. • Explainable AI (XAI) Techniques: Implement advanced techniques like Grad-CAM or LIME to visualize which specific regions of the image the model is focusing on for its predictions. This will build greater trust and interpretability for clinicians, allowing them to understand the AI's reasoning. • Enhanced User Interface and Authentication: Develop a full-fledged, intuitive web-based user interface that includes: o Secure User Authentication: Implement robust login and signup functionalities for medical professionals and authorized users. o Personalized Patient Portals: Create a particular portal for each patient, allowing for organized storage and easy access to their individual scan analyses and reports over time. o Virtual Doctor Consultation Integration: Explore integrating features for virtual doctor advice directly within the web app, facilitating seamless communication and remote consultations based on the AI's analysis. • Continual Learning and Feedback Loop: Implement strategies for the model to continuously learn and improve as more real-world, validated data becomes available through a secure feedback mechanism from clinicians, ensuring the model's performance evolves over time. • Regulatory Compliance: Work towards ensuring the platform adheres to medical device regulations and data privacy standards (e.g., HIPAA, GDPR) for real-world clinical deployment.

Built With

Share this project:

Updates