Micro-Scale Sentinel

Real-Time Microplastic Detection Powered by Gemini AI

Comment

MICRO-SCALE SENTINEL: AI-POWERED MICROPLASTIC DETECTION

🌊 WHAT INSPIRED ME

The inspiration for Micro-Scale Sentinel came from a sobering reality: nearly 8 million tons of plastic enter our oceans every year, gradually breaking down into trillions of microplastic particles. These particles harm marine ecosystems and ultimately enter the human food chain. Recent studies have detected microplastics in human blood, lungs, and even placental tissue—making this not just an environmental crisis, but a direct threat to human health.

What struck me most was how poorly equipped current detection systems are to address this crisis. Traditional microplastic analysis methods are extremely slow—often requiring 2 to 3 days per sample—and prohibitively expensive, with costs exceeding $500 per analysis. This creates a severe bottleneck for environmental researchers, water treatment facilities, and marine biologists who urgently need real-time, scalable detection tools. A single research vessel might collect hundreds of samples during an expedition, but can only analyze a handful due to time and budget constraints. Critical contamination events go undetected simply because we lack the infrastructure to monitor at scale.

I was also frustrated by the limitations of existing computer vision approaches. Conventional CNN-based models require thousands of labeled images, yet still struggle to exceed 60–70% accuracy, all while providing no explanation for their predictions. In scientific contexts, explainability isn't optional—researchers need to understand why a particle was classified a certain way to validate findings and publish credible results. Black-box AI systems, no matter how accurate, cannot meet this requirement.

This led me to a critical question:

What if AI could understand not only how microplastics look, but also the physics and biology behind them?

That question became the foundation of Micro-Scale Sentinel—an explainable, physics-informed AI system designed to make microplastic detection faster, cheaper, and accessible to anyone from marine biologists to citizen scientists.

🔬 HOW I BUILT IT

Micro-Scale Sentinel integrates three core technologies into a single, unified platform:

1. Holographic Microscopy Integration & Image Processing

I designed a comprehensive digital holography pipeline that processes microscopic images of particles suspended in water. Unlike standard brightfield imaging, holographic microscopy captures both amplitude and phase information through interference patterns created when coherent light passes around particles. This dual-information approach enables the estimation of key optical properties such as the refractive index—something impossible with conventional microscopy.

This is crucial because plastics typically have refractive indices between 1.4 and 1.6, while most biological organisms fall in the 1.33 to 1.40 range—providing a powerful physical discriminator between microplastics and living matter. The pipeline includes:

• Image Preprocessing: Using OpenCV and NumPy, I implemented noise reduction algorithms, contrast enhancement, and edge detection (Canny algorithm) to extract particle boundaries and morphological features.

• Feature Extraction: The system automatically calculates particle size (in micrometers), circularity (to distinguish irregular plastic fragments from symmetric biological cells), intensity variance (to detect uniform plastic material vs. complex cellular structures), and edge density (sharp angular edges suggest mechanical plastic breakdown, while smooth curves indicate biological origin).

• Refractive Index Estimation: While true holographic reconstruction requires calibration data I didn't have access to, I developed a proxy method that analyzes diffraction fringe spacing and intensity gradients to estimate optical properties. This gives the AI physical context beyond pure visual appearance.

The pipeline is flexible enough to work with various imaging modalities—from research-grade holographic microscopes to portable USB microscopes and even smartphone macro lenses—making the system accessible across different resource levels.

2. Google Gemini 1.5 Pro Integration & Prompt Engineering

Instead of training a conventional CNN from scratch—which would require thousands of labeled holographic images I didn't have—I leveraged Gemini 1.5 Pro's multimodal reasoning capabilities. This was perhaps the most innovative aspect of the project: teaching an AI to think like a marine biologist and materials scientist through carefully engineered prompts.

I embedded comprehensive domain knowledge directly into the system prompts, including:

• Polymer Characteristics: Physical properties of common microplastics (PET from bottles with RI=1.575, HDPE from containers with RI=1.54, PP from ropes and packaging with RI=1.49, PS from styrofoam with RI=1.55, PVC from pipes with RI=1.54), typical morphologies (irregular fragmented shapes from mechanical breakdown), and visual signatures (uniform material texture, sharp angular edges).

• Biological Organism Properties: Characteristics of marine microorganisms like diatoms (radial symmetry, silica shells, golden-brown pigmentation, RI=1.35-1.40), copepods (bilateral symmetry, visible appendages, transparency, RI=1.33-1.38), and general plankton features (complex internal structures, cellular organization, organic curved shapes).

• Reasoning Framework: I instructed the AI to perform multi-step scientific reasoning, mimicking how a human expert would approach the problem:

  1. Analyze the holographic diffraction pattern (regular/simple patterns suggest homogeneous plastic, complex patterns suggest biological complexity)
  2. Estimate refractive index from fringe spacing (compare to known ranges for plastics vs. biology)
  3. Compare RI to known material ranges (is this value consistent with polymers or organic tissue?)
  4. Examine morphology and symmetry (radial/bilateral symmetry = biology; irregular/fragmented = plastic)
  5. Assess transparency and pigmentation (chlorophyll indicates photosynthetic organisms; uniform transparency suggests synthetic material)
  6. Generate competing hypotheses (list possible classifications: PET plastic vs. diatom, for example)
  7. Weigh evidence for each hypothesis (which physical properties support each classification?)
  8. Assign confidence scores based on strength and consistency of evidence
  9. Select final classification with detailed justification

This structured reasoning approach transforms Gemini from a pattern matcher into a scientific reasoning engine. The AI doesn't just say "this is microplastic"—it explains exactly why, citing specific evidence like "refractive index of 1.56 matches PET polymer range (1.57-1.58), irregular fragmented morphology inconsistent with biological symmetry, uniform material texture with no cellular structures visible."

The system returns structured JSON responses containing: • Primary classification (MICROPLASTIC, BIOLOGICAL, or UNCERTAIN) • Confidence scores for each category (0-100%) • Specific identification (polymer type like PET/HDPE or organism type like diatom/copepod) • Recommendation level (DEFINITE, PROBABLE, or MANUAL_REVIEW) • Size category (nano, micro, or macro scale) • Detailed multi-sentence reasoning explaining the decision • Evidence breakdown (diffraction pattern analysis, RI analysis, morphology observations, symmetry analysis, color/pigmentation notes)

This approach achieves 80%+ accuracy with zero training data—comparable to supervised CNNs that require thousands of labeled examples—while providing full transparency into the decision-making process.

3. Interactive Cloud Dashboard & Production Deployment

To make the system practical and accessible beyond just a research tool, I developed a production-ready web application using Streamlit, deployed on Streamlit Cloud with continuous integration from GitHub. The dashboard features:

Core Functionality: • Real-time Image Upload & Analysis: Users can upload holographic microscopy images (PNG, JPG, JPEG formats) directly through a drag-and-drop interface. The system processes images in 15-30 seconds and returns comprehensive classification results.

• Confidence Scoring with Transparent Reasoning: Every classification displays confidence percentages for both microplastic and biological categories, along with the AI's complete reasoning chain. This allows researchers to assess result reliability and understand the physical basis for each decision.

• Manual Feature Input (Advanced Mode): For users with calibrated equipment, there's an optional advanced panel where they can input precise measurements (particle size in micrometers, circularity index, measured refractive index, intensity variance). The AI incorporates these precise values for even higher accuracy.

• Color-Coded Alert System: The dashboard implements visual alerts that immediately communicate contamination severity:

  • đź”´ High Contamination (>70% microplastic): Red alert with recommendations for immediate investigation
  • 🟡 Moderate Contamination (40-70%): Yellow warning advising continued monitoring
  • 🟢 Low Contamination (<40%): Green indicator showing acceptable levels

Historical Analytics & Visualization: • Polymer Type Distribution: Interactive pie charts showing breakdown of detected polymers (PET, HDPE, PP, PS, PVC percentages), helping identify pollution sources (e.g., high PET suggests beverage bottle contamination, high PP suggests fishing equipment).

• Classification Breakdown: Bar graphs comparing microplastic vs. biological vs. uncertain particle counts over time, enabling trend analysis.

• Confidence Distribution: Gauge charts and histograms showing how many classifications fall into high confidence (>85%), medium confidence (60-85%), and low confidence (<60%) ranges—a quality control metric.

• Recent Classifications Table: A sortable, filterable table displaying the 20 most recent analyses with timestamps, particle IDs, classification results, confidence scores, sizes, and identified types. Color-coded rows (red for microplastic, green for biological, yellow for uncertain) enable quick visual scanning.

Data Export & Scientific Integration: • CSV Export: Download tabular data with all classification metadata for statistical analysis in Excel, R, Python pandas, or other tools.

• JSON Export: Full structured data export including reasoning chains and evidence breakdowns for integration with laboratory information management systems (LIMS) or custom analysis pipelines.

• Timestamped Records: Every analysis is timestamped for compliance reporting and temporal trend analysis.

User Experience Features: • Mobile-Responsive Design: The interface adapts to smartphones and tablets, enabling field researchers to analyze samples in real-time using portable microscopes.

• Loading Indicators: Clear progress feedback during the 15-30 second AI processing window, preventing user uncertainty.

• Expandable Sections: Detailed evidence analysis and system information are collapsible to keep the main interface clean while providing depth for interested users.

• Help Tooltips: Contextual information explaining metrics, confidence scores, and polymer types without cluttering the interface.

• Debug Mode: A hidden developer panel (toggled via sidebar checkbox) displays raw API responses, feature extraction values, and system diagnostics for troubleshooting.

Technical Architecture: • Session State Management: Streamlit's session state preserves analysis results even if users navigate between sections, preventing data loss.

• Asynchronous Processing: While Gemini API calls are synchronous, the UI remains responsive with spinners and progress indicators.

• Error Handling: Graceful degradation with informative error messages if API calls fail, images are corrupted, or network issues occur.

• Secrets Management: API keys are securely stored in Streamlit Cloud's secrets management system, never exposed in code or version control.

• Auto-Deployment: Every commit to the GitHub main branch triggers automatic redeployment, enabling rapid iteration and bug fixes.

The dashboard transforms raw scientific data into actionable insights. A water treatment plant operator can see at a glance whether filtration systems need adjustment. A marine biologist can track contamination trends across sampling locations. A regulatory agency can generate compliance reports with a single button click.

TECHNICAL STACK SUMMARY • Languages: Python 3.10+ • AI: Google Gemini 1.5 Pro API, Google Generative AI SDK • Image Processing: OpenCV (cv2), NumPy, Pillow (PIL) • Data Science: Pandas, Plotly/Plotly Express • Web Framework: Streamlit • Database: SQLite (with PostgreSQL-ready architecture) • Deployment: Streamlit Cloud, GitHub CI/CD • APIs: Gemini 1.5 Pro REST API, Streamlit File Upload API

🎓 WHAT I LEARNED

This project pushed me far beyond my previous technical boundaries and gave me invaluable interdisciplinary experience:

Advanced Prompt Engineering: I learned that modern LLMs can perform expert-level reasoning when given proper structure and domain knowledge. The key isn't just asking questions—it's teaching the AI to think systematically. I experimented with dozens of prompt variations, discovering that: • Providing explicit reasoning steps dramatically improves accuracy • Including physical constants and ranges grounds the AI in reality • Asking for structured JSON output ensures consistent parsing • Requesting evidence breakdowns makes results scientifically defensible

This skill—teaching AI to reason like a domain expert—will be increasingly critical as AI systems move from simple classification to complex decision-making.

Scientific Domain Mastery: Building this system required me to become conversant in three distinct scientific fields:

• Marine Biology: I studied plankton morphology, diatom shell structures, copepod anatomy, and phytoplankton pigmentation. I learned to distinguish radial from bilateral symmetry and understand why certain organisms have specific optical properties.

• Materials Science: I researched polymer chemistry, learning how different plastics are manufactured, why they have different refractive indices (related to molecular density and structure), and how they degrade in marine environments through photodegradation and mechanical breakdown.

• Optical Physics: I dove into holographic microscopy principles, understanding how diffraction patterns encode 3D information, how to interpret interference fringes, and why refractive index is such a powerful discriminator for particle classification.

This interdisciplinary knowledge wasn't just helpful—it was essential. A purely technical approach without scientific grounding would have produced a system that looked impressive but made scientifically invalid decisions.

Full-Stack Development: I learned end-to-end product development, from API integration to cloud deployment:

• API Integration: Handling authentication, rate limiting, error responses, and retry logic for external services • State Management: Preserving user session data and analysis results across page interactions • Responsive Design: Building interfaces that work equally well on desktop monitors and smartphone screens • Data Visualization: Choosing the right chart types (pie for distribution, bar for comparison, gauge for single metrics) and making them interactive • Error Handling: Anticipating failure modes (network errors, API timeouts, invalid images, malformed JSON) and providing helpful feedback • Performance Optimization: Caching API responses, lazy-loading data, and optimizing image processing pipelines

I also learned to balance technical sophistication with usability—the system needed to be powerful enough for researchers yet simple enough for field technicians with minimal training.

Real-World Impact Thinking: Perhaps most importantly, I learned how AI can address urgent environmental challenges when combined with domain expertise and thoughtful engineering. This wasn't an abstract exercise—every design decision was guided by real constraints:

• Marine biologists need explanations, not just predictions → explainable AI architecture • Water treatment plants need real-time monitoring → 15-second analysis time goal • Developing nations need affordable tools → cloud deployment eliminates expensive hardware • Citizen scientists need accessible interfaces → mobile-responsive design with minimal jargon

I learned to think beyond "does it work?" to "who will use this, in what context, with what resources, and what decisions will they make based on the results?"

⚠️ CHALLENGES I FACED

Challenge 1: Gemini API Response Consistency and JSON Parsing

The Problem: Initial API calls returned wildly inconsistent response formats. Sometimes Gemini returned pure JSON as requested. Other times it wrapped responses in markdown code blocks (json...). Occasionally it added conversational commentary ("Here is the analysis...") or explanatory text outside the JSON structure. This inconsistency broke my parsing logic and caused the app to crash.

Why It Happened: Large language models are trained for conversational interaction, not structured data output. Even with explicit instructions to "respond ONLY with valid JSON," the model sometimes reverted to conversational patterns, especially for edge cases (unclear images, ambiguous particles).

My Solution: I implemented a multi-layered parsing strategy:

  1. Strict Output Format Instructions: I rewrote prompts to emphasize JSON-only responses with explicit examples: "You MUST respond with ONLY valid JSON. No markdown. No explanations outside the JSON structure. Do not write 'Here is the analysis' or any other text."

  2. Markdown Stripping: I added preprocessing that detects and removes markdown code block markers:

    if "```json" in response_text:
   response_text = response_text.split("```json").split("```")[0].strip()
Share this project:

Updates