Project Story

About the Project

This project, HVAC Energy Savings Predictor, was built to explore how Reinforcement Learning (RL) can intelligently optimize HVAC (Heating, Ventilation, and Air Conditioning) systems to save energy without compromising indoor comfort.
It uses real-time weather forecasts to simulate the environment and predicts the energy efficiency and comfort performance of three systems:

• Traditional HVAC
• Rule-based Advanced HVAC
• AI-powered RL Agent (SAC model)

The frontend provides a simple, intuitive way for users to click a button and instantly see their personalized HVAC energy savings based on their current weather conditions.

Inspiration

The inspiration came from recognizing how much energy HVAC systems consume globally — they are responsible for nearly 40% of building energy use.
With the rapid evolution of RL algorithms and the growing need for sustainable technology, it felt natural to combine real-time weather intelligence with a trained RL agent for smart HVAC control.

How I Built It

• AI Model Training (Core Focus):

  • Designed a custom SensorBasedThermalEnv Gym environment that accurately simulates indoor thermal dynamics based on outdoor temperature, wind speed, and humidity.
  • Trained a Soft Actor-Critic (SAC) Reinforcement Learning agent using the simulated environment.
  • The SAC agent learned an optimal HVAC control policy balancing energy efficiency and comfort penalties over time.

• Environment and AI Integration:

  • Carefully structured environment state space and reward functions to reflect real-world physics: temperature drift, HVAC cooling/heating effects, and random noise factors.
  • Tuned training hyperparameters such as learning rate, reward scaling, and action noise for stable and effective policy learning.

• Model Deployment:

  • Exported the trained SAC agent using stable-baselines3 saving mechanisms.
  • Integrated the agent inside a FastAPI backend server, allowing real-time inference based on fresh weather data fetched from the OpenWeatherMap API.

• Baseline Comparisons:

  • Developed a ComparisonModule to benchmark the AI agent's performance against:
  • A traditional fixed setpoint controller
  • A manually tuned rule-based advanced controller

• Frontend Interface:

  • Built a simple HTML+JavaScript frontend.
  • Used Chart.js for interactive energy consumption visualization and savings comparison.

What I Learned

• Reinforcement Learning Deployment: Not just training an RL agent, but making it inference-ready, lightweight, and callable via APIs.
• Custom Gym Environment Design: Simulating semi-realistic indoor environments that accurately react to control signals and external conditions.
• FastAPI Backend Architecture: Building APIs that tightly couple AI models with real-time external data sources.
• Frontend-Backend Communication: Handling CORS issues and asynchronous API calls properly to bridge the UI and AI logic.
• Model-Environment Synchronization: Ensuring training and testing environments matched closely for successful deployment.

Challenges Faced

• Environment-Model Mismatch:
  Designing a simulation environment that was sufficiently realistic but computationally light was a major challenge. Small mismatches in temperature drift modeling could destabilize the RL agent during deployment.

• Reward Engineering: Defining a reward function that properly balances energy savings with thermal comfort required careful trial and error. Too much penalty weight on either side led to unrealistic behaviors (either freezing or overheating).

• SAC Algorithm Stability: The Soft Actor-Critic algorithm, while powerful, was sensitive to hyperparameters like entropy coefficient and target smoothing. Multiple retraining runs were required to achieve stable policies that generalized across weather scenarios.

• Model Inference Optimization: Loading a full RL agent and running it for every API request could be slow if not handled carefully. Optimizations around loading the model once at server startup and caching helped improve responsiveness.

• Weather Data Noisiness: Weather API forecasts had some inconsistencies (e.g., sudden unrealistic jumps in humidity or temperature). The environment needed to smooth and preprocess this data before feeding it to the AI agent.

Conclusion

This project demonstrates the power of integrating Reinforcement Learning, real-time data streams, and API-driven architecture to create intelligent, adaptive systems for sustainability. It provides a glimpse into how AI can revolutionize traditional building control strategies, achieving both energy efficiency and human comfort in dynamic real-world environments.

Built With

• gym
• python
• pytorch