EcoLens: Intelligent Carbon Scanner

Turning messy shopping receipts into actionable climate data instantly using Gemini Flash Multimodal reasoning.

Comment

💡 Inspiration

Climate change is the defining crisis of our time. While many people want to reduce their carbon footprint, tracking it is incredibly difficult. Most carbon calculators only focus on transport (flights/cars), ignoring a huge chunk of daily emissions: Retail Consumption (FMCG).

We realized that shopping receipts hold the key to this data. However, manually inputting every item from a grocery run into a carbon calculator is tedious and creates high friction. We needed a way to bridge the gap between physical paper receipts and digital carbon data instantly.

🤖 What it does

EcoLens is an AI-powered scanner that transforms a photo of any shopping receipt into a detailed carbon emission report.

  1. Snap: User takes a photo of a receipt (even if it's crumpled or uses abbreviations).
  2. Analyze: Gemini 2.5 Flash processes the image to extract product names, creates context for abbreviations (e.g., identifying "Sari Roti" as a grain product), and categorizes items based on scientific data.
  3. Reveal: The app displays the estimated carbon footprint of that shopping trip in seconds.

⚙️ How we built it

We built EcoLens using Flutter for the mobile frontend and Google Gemini 2.5 Flash via the google_generative_ai SDK and REST API.

Instead of a traditional pipeline (OCR $\rightarrow$ Regex $\rightarrow$ Database Search), which is prone to errors with Indonesian receipt formats, we utilized Gemini's native multimodal capabilities.

We implemented a robust AI pipeline that performs:

  1. Optical Character Recognition (OCR): Reading text from low-quality thermal paper images.
  2. Semantic Classification: Deciphering non-standard abbreviations (e.g., mapping "Indomilk Choco" to "Processing of Food products nec").
  3. Carbon Mapping: Assigning the item to the closest scientific industrial sector based on global EEIO (Environmentally Extended Input-Output) standards used by environmental economists.

The Logic

We utilize Gemini to ensure structured output. The calculation logic followed the Spend-Based Method, utilizing a multi-regional input-output database to derive emission factors adjusted for currency and inflation:

$$TotalEmission = \sum (Price_{IDR} \times CurrencyRate \times InflationFactor \times EmissionFactor_{sector})$$

Here is a snippet of how we prompt Gemini to ensure consistent classification without hallucination:

// We ask Gemini to act as an Environmental Economist
final prompt = """
IMPORTANT: You are an expert environmental economist.
Classify the product based *solely* on its name into ONE of the specific economic sectors defined in the standard EEIO industrial classification system.
...
Examples:
- Input: "Coca Cola", Output: "Manufacture of beverages"
- Input: "Aqua 1.5L", Output: "Collection, purification and distribution of water"
""";

🧠 Challenges we faced

  • Receipt Quality: Indonesian receipts are often faded, crumpled, or use mixed languages (Bahasa Indonesia & English). Traditional OCR libraries failed to capture the context, but Gemini's vision capabilities handled this surprisingly well.
  • Hallucination: Initially, the AI would guess carbon values randomly. We solved this by restricting Gemini only to categorize items (returning the Sector Name), while the actual math happens deterministically in our application code using a local JSON database of emission factors.
  • Latency & Limits: We needed the experience to feel instant. We optimized the maxOutputTokens and utilized Gemini 1.5 Flash which significantly reduced inference time compared to other models.

🏆 Accomplishments that we're proud of

  • Successfully reducing the time to track a weekly grocery shop from 10 minutes (manual entry) to under 5 seconds.
  • Achieving high accuracy in categorizing ambiguous items using Gemini's context window.
  • Building a Safe Parsing mechanism that prevents the app from crashing even if the AI returns incomplete data or hits safety filters.

🚀 What's next for EcoLens

  • Personalized Recommendations: Using Gemini to suggest lower-carbon alternatives for specific items found in the receipt.
  • Gamification: Turning the carbon data into a "Green Score" to motivate users.
  • API Integration: Opening this receipt-to-carbon API for other developers.

Built With

Share this project:

Updates