EcoVoice

Azure AI powered sustainability coach

Comment

EcoVoice: Your Personal Sustainability Coach

Inspiration

In a world increasingly threatened by climate change, we realized that individual actions matter most. Many people want to be environmentally conscious but lack personalized guidance and motivation. EcoVoice was born from the vision of transforming sustainability from an overwhelming concept to an engaging, accessible journey. By leveraging cutting-edge Azure AI technologies, we created a platform that turns eco-friendly living into an interactive, gamified experience that speaks directly to each user's unique lifestyle and choices.

What it does

EcoVoice is a multi-modal web application that acts as a personal sustainability coach, helping users understand and improve their environmental impact through:

  • Text Analysis: Users log daily activities, which are analyzed for sustainability potential
  • Image Recognition: Scan groceries, waste, and daily items to get eco-friendly recommendations
  • Voice Insights: Spoken reflections provide deeper context to sustainability efforts
  • Gamification: Earn "Eco Points" for sustainable choices, creating positive reinforcement
  • Personalized Recommendations: Azure OpenAI-driven, tailored advice for reducing carbon footprint

The app transforms abstract environmental concepts into concrete, actionable steps unique to each user's lifestyle. How we built it We constructed EcoVoice using a modern, scalable architecture powered by Azure services:

Highlevel

alt text

Key Components

Frontend

  • React App hosted on Azure Static Web App
  • Integrates with Azure AD for user authentication
  • Uses Azure Content Safety for content moderation

Backend

  • **.C# Azure Functions for serverless compute
  • .NET Function App with three main API endpoints:
    • AnalyzeUserInput: Processes text, image, and voice inputs
    • GetUserProfile: Retrieves user profile information
    • UploadFile: Handles file uploads to blob storage
  • Independently scalable from the frontend

Data Flow

  1. Text Analysis Flow:

    • User enters text in the React app
    • Frontend sends text directly to AnalyzeUserInput API
    • Backend processes text through Azure Cognitive Services and OpenAI
    • Results are stored in Cosmos DB and returned to the user

  2. Image Analysis Flow:

    • User uploads an image in the React app
    • Frontend sends image to UploadFile API
    • Backend stores image in Azure Blob Storage (Images container)
    • Frontend requests analysis via AnalyzeUserInput with the blob reference
    • Backend processes image through Azure Cognitive Services and OpenAI
    • Results and blob reference are stored in Cosmos DB and returned to the user

  3. Voice Analysis Flow:

    • User records audio in the React app
    • Frontend sends recording to UploadFile API
    • Backend stores audio in Azure Blob Storage (Audio container)
    • Frontend requests analysis via AnalyzeUserInput with the blob reference
    • Backend processes audio through Azure Cognitive Services and OpenAI
    • Results and blob reference are stored in Cosmos DB and returned to the user

Data Storage

  • Azure Cosmos DB stores:
    • User profiles in the "users" container
    • Activity records in the "activities" container (including blob references)
  • Azure Blob Storage stores:
    • Images in the "images" container
    • Audio recordings in the "audio" container

Security

  • All API calls are secured with Function Keys
  • User authentication is handled by Azure AD
  • Content moderation is performed by Azure Content Safety

Azure Cognitive Services for multi-modal analysis:

  - Computer Vision for image recognition
  - Text Analytics for activity insights
  - Speech-to-Text for voice input processing

Azure OpenAI service for chat completion

gpt40mini

Azure Content Safety Integration (One of Microsoft's Responsible AI tools):

EcoVoice leverages Azure AI Content Safety service to create a safe, respectful environment for all users. This critical integration:

  • Screens user-submitted text for harmful content
  • Ensures community guidelines are automatically enforced
  • Protects users from exposure to inappropriate material
  • Maintains the integrity of our sustainability platform

Data Management:

  • Azure Blob Storage for video and image blobs Secure, scalable cloud storage solution
  • Azure Cosmos DB for user data and analysis results along with blob content identifier.

Authentication:

Azure Active Directory B2C for user management and security Azure function key

This application uses Microsoft Azure AD for secure user authentication. When logging in:

  • Users will be prompted to sign in with their Microsoft account
  • The app only requests basic profile information
  • "Unverified" label may appear until our publisher verification is complete
  • We do not access emails, contacts, or other sensitive Microsoft account data

image

Users concerned about permissions can revoke access anytime through their Microsoft account settings (account.microsoft.com → Privacy → Apps and services).

image

Technical Backbone

Azure AI services – text, speech and image analysis

Azure Open AI services – chat completion to get personalized inputs

Azure content safety service

Azure static web apps to host React UX

Azure function app to host backend

Azure Cosmos DB- to store user and activities

Azure AD integration – to manage user identities

Azure storage – to store image and voice blob

CICD automation

GitHub workflow

Challenges we ran into

Building EcoVoice wasn't without its hurdles:

  • Data Integration: Synchronizing insights from text, images, and voice inputs required complex AI modeling
  • Personalization at Scale: Creating meaningful, individualized recommendations across diverse user profiles
  • Performance Optimization: Ensuring fast, responsive analysis within Azure free-tier constraints
  • AI Accuracy: Training models to provide genuinely helpful, context-aware sustainability advice
  • User Engagement: Designing a gamification system that motivates without feeling patronizing
  • Audio capturing: We had issues with Audio component in the frontend. Copilot was very helpful in providing potential suggestions to fix those errors.

Accomplishments that we're proud of

Despite the challenges, we achieved remarkable milestones:

  • Successfully integrated multiple Azure AI services into a cohesive platform
  • Created a truly multi-modal sustainability coaching experience
  • Developed an intuitive, gamified interface that makes eco-friendly living fun
  • Demonstrated scalable serverless architecture using Azure Functions
  • Proved that technology can drive personal environmental consciousness

What we learned

Our EcoVoice journey was a profound learning experience:

  • Deep insights into Azure's cognitive services and AI capabilities
  • Advanced techniques in multi-modal data processing
  • Serverless architecture design and optimization
  • User experience design for complex AI-driven applications
  • The power of gamification in driving behavioral change

Leveraging Azure Free Credit for Full-Scale Development

"I strategically selected Azure services that are available in either free or low-cost tiers. This thoughtful resource planning allowed me to develop, test, and deploy my application entirely within my Azure free tier credit allocation, making the project both technically robust and cost-efficient during the development phase." This emphasizes the deliberate selection process and highlights how it enabled complete project delivery within free tier limitations.

What's next for EcoVoice

The journey doesn't stop here! Future roadmap includes:

Advanced machine learning models for even more personalized recommendations Community features to create collaborative sustainability challenges Integration with smart home and IoT devices Expanded language and regional sustainability insights Carbon footprint tracking and detailed environmental impact reporting

EcoVoice isn't just an app—it's a movement towards more conscious, sustainable living, powered by the incredible potential of Azure AI.

Built With

Share this project:

Updates