TRACK: Earth and Space
OPT-IN: GitHub, Redis Cloud
Inspiration
We wanted to build a product with the biggest impact on saving the environment and encouraging sustainable practices. To do so, we looked to enterprise companies and agricultural hubs because they are responsible for the vast majority of greenhouse gas emissions (71%!).
Companies want to reduce their emissions for a variety of reasons, including regulatory compliance, cost savings, risk mitigation, brand reputation, and investor/shareholder expectations. The problem is that managing sustainability campaigns and practices is simply too hard - on average, top companies use an average of 37 different software tools in their day to day. There’s too much data in too many different places, and enterprises can’t use the data they want to analyze where and how exactly they can be more environmentally sustainable.
What it does
EcoSense AI is a web application that lets companies instantly connect to over 100+ of the software tools they use. For enterprise companies, this could be anything from product lists, emission reports, customer reports, project management tasks, etc - the possibilities are endless. Specifically for agricultural hubs, EcoSense AI allows companies to import location data which is then turned into satellite imagery, and import simulated IOT sensor measurements such as soil moisture, humidity, temperature, and rainfall.
EcoSense AI automatically pulls in all of the data from relevant sources, and presents full analysis and daily PDF reports to the company regarding its sustainability practices and how they can improve.
Companies can then ask EcoSense AI relevant questions regarding their environmental concerns. EcoSense AI takes the full context of the data into consideration and generates optimal responses to queries.
Finally and most importantly, EcoSense AI generates specific actionables that the company should take in the context of the data and analysis. These actions include sending emails to employees, adding tasks into work management software, creating calendar events, and more. Companies can work with the EcoSense AI to edit the actionables to suit their specific needs, and when approved, EcoSense AI automatically executes the actions on behalf of the company.
How we built it
EcoSense AI is built on a Python Flask backend framework with HTML/CSS/JS and Jinja templating on the frontend.
We use Endgrate and custom defined data schemas for resources in order to pull in data from different sources for analysis, and push out data when action plans are executed. For pulling in data, we use Ngrok to forward our webhook endpoint to the cloud. We have a custom hosted Azure OpenAI instance with a Redis prompt cache to run LLM queries (all queries are first SHA256 hashed + cached), and a custom hosted Azure Cognitive Services instance to run image analysis ML on satellite images obtained through geocoding an arbitrary user-specified location and then querying the Google Satellite Map API. We use numpy and matplotlib for simulating/graphing IOT data and reportlab for generating PDFs.
The codebase is hosted on GitHub, where we extensively use commits, PR, issues, and encourage other companies to collaborate by submitting PRs for new resource schemas.
Challenges we ran into
This was not an easy project to build. Among many other roadblocks, we found that…
- NASA’s Earth API is currently completely down and not functional. We were heavily relying on the API for satellite imagery and fell into a rabbit hole of searching for alternatives that did not work (Azure’s maps API only returned non-satellite map images, USGS and other government tools had images that were completely covered by clouds, etc.) until we landed on the Google Maps API.
- Harvard’s wifi firewall blocks access to Redis cloud instances, ONLY if they are hosted on AWS. We were extremely stumped on this problem for a long, long time only to discover it worked completely fine if we hosted the instance on Azure.
- Matplotlib has issues working with threads across different OS. We were developing on both Linux and macOS so this issue was incredibly confusing.
- Countless bugs.
Accomplishments that we're proud of
We are very proud of the fact that we built this project from start to finish in a weekend, and have an awesome MVP to demo the power of integrations when combined with AI, solving a critical problem and having a real-world impact.
What we learned
This hackathon has furthered our belief that Endgrate is, without a doubt, the future of integrations development.
What's next for EcoSense AI
Next up, we hope to:
- Multithread the entire application with gunicorn, and use grequests to multithread API calls for faster performance.
- Have AI autogenerate more charts + data visualizations that can be useful.
- Add more action types.
- Convince some sponsors to allow us to host the site and make it completely free for any company to use.
Log in or sign up for Devpost to join the conversation.