.env

.env uses AI to predict produce spoilage, cutting food waste and GHG emissions 40%. Our ML model detects invisible ripeness markers, ensuring safe consumption and brings surplus food to communities.

Comment

Our Inspiration

We were inspired to start this project after learning about the staggering scale of global food waste. Every year, 1.3 billion tons of food are wasted globally, with fruits and vegetables accounting for a significant portion. With a global goal to cut food waste in half by 2030 and knowing that households are responsible for 43% of this waste, we wanted to find a way to empower everyday consumers to make a difference. Our goal was to democratize produce waste reduction by putting a powerful tool right into the hands of the people who could use it most.

Some Context

Previously members of our team performed research in this field where they developed methods to determine ripeness of fruits. The application of the methods and the model were done this weekend at PennApps. We are including the research methodology to explain how our proprietary model works!

What We Learned

Our initial research led us to Hyperspectral Imaging (HSI), a technique that captures image data across the electromagnetic spectrum, going beyond the red, green, and blue (RGB) light our eyes see. We learned that HSI is very effective at determining the ripeness of produce by analyzing its unique spectral signature. For example, an overripe tomato reflects light differently than a perfectly ripe one, which can be seen in their different spectral plots.

However, the biggest hurdle was that commercial HSI cameras are incredibly expensive, costing thousands of dollars, making them inaccessible for regular consumers or even most grocery stores. This led us to our core research question: Could we combine standard RGB imaging from a smartphone with a specific slice of the infrared (IR) spectrum to predict ripeness affordably?

How We Built Our Project

Our project evolved through a multi-step experimental process, from data collection to full-scale app development with real-world deployment challenges.

1. Data Collection & ML Foundation

First, we needed to build our own dataset. We collected images and data from over 200 tomatoes. Using an iPhone equipped with a simple, low-cost 720nm IR-pass filter, we captured both a standard RGB image and a near-infrared (IR) image of each tomato. Simultaneously, we measured the actual firmness of each tomato in Newtons (N) using a scientific instrument called a penetrometer to get a ground-truth value for its ripeness.

2. Machine Learning Models

With the data collected, we developed two different machine learning (ML) models to predict firmness from the images:

  • Model 1: This model used the average pixel intensities from both the RGB and the near-IR images as features to predict the firmness value.
  • Model 2: This was a more advanced approach using a Pix2Pix architecture, which is a type of conditional generative adversarial network (cGAN). This powerful model learned how to generate a synthetic IR image directly from a standard RGB photo, meaning a user wouldn't even need a special filter.

3. Cross-Platform App Development Journey

What started as a simple iPhone app concept evolved into a comprehensive cross-platform ecosystem built with Flutter. This decision came with its own set of technical challenges and breakthroughs:

  • Framework Selection: We chose Flutter 3.13+ with Dart for true cross-platform development, enabling us to serve both mobile users and web-based suppliers with a single codebase while maintaining native performance.
  • Real-Time Backend Integration: We implemented Supabase as our real-time database solution, allowing instant synchronization between consumer produce scans and supplier analytics dashboards.
  • Advanced Visualization Challenges: We pivoted from Python's matplotlib to native Flutter charts using fl_chart, which provided better performance and seamless integration.
  • Geographic Heatmap Innovation: We innovated a solution using concentric circles with decreasing opacity levels, creating a weather map-style gradient effect that beautifully shows quality distribution across regions.
  • Authentication & Route Management: After facing API compatibility issues, we successfully implemented route-aware background selection using GoRouterState.of(context).uri.toString().
  • Platform-Specific Asset Management: We developed a system where the web application uses specific backgrounds while mobile maintains its own visual identity, all managed through platform detection using kIsWeb.
  • Location Services Integration: Adding user location functionality to the heatmap required implementing comprehensive permission handling, location services management, and smooth camera animations using the geolocator package.
  • Data Localization: We strategically relocated our data concentration from California to Philadelphia, requiring updates to both our data population scripts and default map centering logic.

Challenges We Faced

Technical Hardware Constraints

The main challenge was finding an affordable and effective alternative to expensive HSI cameras. We tested five different IR-pass filters, but we quickly discovered that only the 720nm filter produced sharp, high-fidelity images. The images from the other four filters were blurry and unusable for the study. This is regarding the prior research.

ML Model Accuracy

Another challenge was ensuring the accuracy of our predictive models. After extensive training and testing, our models achieved a Root Mean Square Error (RMSE) between 2.8N and 3.1N when comparing the predicted firmness to the actual measured firmness. This showed a very strong correlation.

Software Development Hurdles

  • API Integration Complexity: Integrating multiple AI services (Google Gemini, our custom GCP model, and Cerebras LLaMA) required careful orchestration of parallel API calls and robust error handling.
  • Real-Time Data Synchronization: Ensuring immediate updates in supplier dashboards required implementing sophisticated database triggers and row-level security policies in Supabase.
  • Performance Optimization: Balancing feature richness with app performance meant implementing lazy loading, background processing, and smart caching strategies.
  • Cross-Platform UI Consistency: Achieving a pixel-perfect design across web browsers, iOS, and Android required extensive testing and platform-specific optimizations.

Technical Innovation & Impact

Our development journey resulted in several technical innovations:

  • Dual-API Intelligence: Combining Google Gemini's brand detection with our proprietary ripeness prediction creates a comprehensive produce analysis system.
  • Adaptive Heatmap Visualization: Our custom gradient circle approach provides superior geographic data visualization compared to standard mapping solutions.
  • Context-Aware AI Assistant: AskEnv leverages produce analysis results to provide personalized recipe and sustainability recommendations.
  • Real-Time Supply Chain Intelligence: Suppliers gain immediate visibility into produce quality distribution across their distribution network.

Ultimately, this research and development journey demonstrated multiple successful and economical methods for predicting produce ripeness and resulted in a comprehensive platform that helps consumers, suppliers, and communities reduce food waste. Our next steps include expanding to other commonly wasted items like avocados, integrating additional imaging techniques for improved accuracy, and scaling our platform to serve food banks and retailers directly.

The technical foundation we've built, from advanced ML models to sophisticated cross-platform applications, proves that consumer-grade technology can address global sustainability challenges when combined with thoughtful design and robust engineering.

What's Next For .env?

Well - let's build a greener planet one step a time! Or maybe, one fruit at a time!

Built With

  • 720nm
  • android
  • camera/image-picker
  • cerebras-llama-3.3-70b
  • dart-3.1+
  • fl-chart
  • flutter-tts
  • geolocator
  • go-router
  • google-ai-platform-databases:-postgresql-(supabase)
  • google-cloud-platform-cloud-services:-google-cloud-functions
  • google-maps-platform
  • google-places
  • html/css-frameworks:-flutter-3.13+
  • ir
  • javascript
  • jwt-auth
  • material-design-3
  • opencv
  • provider-pattern
  • python-3.8+
  • python-flask-platforms:-ios
  • row-level-security
  • sharedpreferences-apis:-google-gemini-vision
  • speech-to-text
  • sqlite
  • supabase
  • supabase-rest/realtime-technologies:-custom-cgan-(pix2pix)
  • tensorflow/pytorch
  • web-(pwa)
Share this project:

Updates