Inspiration
Decarbonizing our economy requires smarter use of energy, and buildings play a huge role. Residential and commercial buildings consume roughly 30% of global energy and generate ~27% of greenhouse emissions. Accurately predicting a building’s short-term electricity demand (load forecasting) lets us schedule renewables and shift loads to reduce emissions. For example, knowing tomorrow’s demand allows utilities to incentivize reduced use during peaks or to dispatch clean energy when needed. The growth of rooftop solar, batteries, and electric vehicles adds variability: homes can produce or store energy, making demand patterns complex. This complexity motivates a flexible, data-driven solution. Modern machine learning excels at finding patterns in complex sequences. Recently, Time Series Foundation Models (TSFMs) have emerged: large, pre-trained forecasting networks that generalize across domains. These TSFMs promise accurate day-ahead load forecasts and even anomaly detection for new buildings without retraining, helping building operators integrate renewables and cut fossil use.
What it does
Our solution leverages a pre-trained TSFM to perform two key tasks: short-term load forecasting and anomaly detection for building energy use. Given a history of a building’s demand (and optional exogenous inputs like weather), the TSFM produces an hour-by-hour forecast for the next day. This forecast helps energy managers adjust HVAC schedules, charge EVs, or run batteries when costs or emissions are lowest. Because the model is pre-trained on diverse data, it can be applied to a new building out-of-the-box. In practice, the model takes the past 𝑤 hours of load and predicts the next 𝐻 hours. We train the model by minimizing a loss like Mean Squared Error (lower MSE means better forecasts). After training, the model is run on live data to forecast each building’s demand. Any large deviation between actual load and forecasted load is flagged as an anomaly (e.g. unexpected HVAC fault or unplanned usage spike). TSFMs have been shown to detect such deviations by comparing incoming data to their learned “normal” load patterns.
How we built it
We implemented the forecasting system using a Transformer-based TSFM architecture (similar to Google’s TimesFM). Key steps include:
Data Collection and Preprocessing: We gather a large corpus of building energy time series. To compensate for scarce real-world data, we first pre-train on simulated building models. In fact, NREL’s BuildingsBench provides ~900,000 simulated building-hour series (covering varied US building types) for pretraining. We also curate real building datasets (apartments, offices, schools) for evaluation. All data are cleaned (e.g. linear interpolation of short gaps) and normalized. Exogenous features like outdoor temperature or day-of-week are aligned with the time series.
Model Architecture: We use a decoder-only Transformer as the core TSFM. Historical loads (and any features) are split into short patches and embedded into tokens. These tokens pass through stacked self-attention and feed-forward layers, capturing long-range and seasonal patterns. The model is trained to autoregressively predict the next output patch (set of future hours), reducing roll-out errors. Compared to text models, we emphasize output length over vocabulary. In practice, we base our implementation on existing open-source TSFMs (e.g. TimesFM with ~200M parameters) and the NeuralForecast library for Transformers.
Pre-Training: We first train the TSFM on the full BuildingsBench simulation dataset. This step teaches the model common load patterns (daily cycles, weekly cycles, weather effects) across many building types. The optimization minimizes forecast error on the next day given a week’s history. By the end, the model has seen billions of training points, acquiring generalized temporal knowledge.
Fine-Tuning: For specialized use cases (e.g. a data center vs a school), we optionally fine-tune the model on a small amount of real building data. This adapts the model to specific climate or operational patterns. In practice we found that even without fine-tuning, the model often predicts well; fine-tuning simply refines accuracy.
Deployment: In production, the model runs in inference mode. Every day (or hour), we feed the latest 24–168 hours of a building’s metered load into the model to forecast the next 24 hours. These forecasts feed into energy management decisions (e.g. when to run on-peak appliances or charge storage). Simultaneously, we compute the real-time forecast error for anomaly detection: if it exceeds a threshold (set via historical variance), the system alerts facility staff.
Throughout development, we used PyTorch and HuggingFace for the transformer code, and scaled training on GPUs using NVIDIA’s NeMo or similar frameworks. We evaluated accuracy using standard metrics (MAE, MAPE, RMSE) and compared against baseline models like ARIMA and vanilla LSTM.
Challenges we ran into
Building load forecasting presents several difficulties. First, data scarcity is a major challenge: many buildings do not have years of labelled consumption data. This lack motivated our use of synthetic data. As NREL notes, “current AI approaches for STLF are hindered by a lack of data”, which is why we leveraged large simulated datasets (BuildingsBench) to pretrain our model. Another challenge is domain shift: simulated or aggregated data may not perfectly match a new building’s behaviour. For instance, occupancy schedules or unusual equipment (like EV chargers) can differ. We saw that input features matter: like other studies, we found performance sensitive to including relevant exogenous variables (e.g. including accurate weather forecasts significantly improved predictions).
Training large TSFMs itself was computationally intensive. Even a “small” 100M-parameter transformer requires hours of GPU time on billions of time steps. We had to balance model size against latency: too large and inference is slow for real-time use. Another hurdle was model interpretability: stakeholders wanted to understand why a forecast was made. We addressed this by tracking attention scores and visualizing forecast vs actual trends. Lastly, anomaly detection can be tricky – false positives occur if the model hasn’t seen similar spikes. We mitigate this by tuning thresholds and allowing the model to incorporate recent deviations as context.
Accomplishments that we're proud of
High Forecast Accuracy: Our TSFM-based pipeline achieved state-of-art accuracy on test sets. In benchmarks, the TSFM matched or exceeded specialized Transformers trained from scratch. For example, TimesFM (our base model) outperformed all scratch-trained Transformer models on longer historical windows, confirming its strength for daily forecasts. Even without retraining, it gave competitive zero-shot predictions, in line with Google’s findings that “TimesFM… displays impressive zero-shot performance” on new datasets
Data Efficiency: By using a foundation model, we dramatically reduced the need for per-building training. In experiments, we found the model worked well with no local data (zero-shot) and needed only a few days of local load to adapt further. This efficiency matches prior work: “foundation models perform comparably to trained-from-scratch Transformers… without domain-specific training”
Anomaly Detection Capability: We successfully integrated anomaly detection on top of forecasting. In trials, the system correctly flagged issues like malfunctioning HVAC or unplugged equipment by detecting abnormal demand spikes. This dual use (forecasting + anomaly detection) is a bonus of TSFMs, which “identify deviations from expected patterns by comparing current signals with their learned representation of normal behaviour”. Building owners found these alerts valuable for maintenance and energy savings.
What we learned
This project taught us several lessons. First, transformers are indeed state-of-the-art for load forecasting: univariate Transformer models consistently beat traditional approaches like ARIMA or LSTMs on this problem. We saw that the model learned daily and weekly cycles that generalize across buildings. Second, pretraining on simulated data can be very effective. By training on BuildingsBench’s 900k-building dataset, the TSFM captured common load “grammar” (time-of-day patterns, weather effects) that transferred to real buildings. In hindsight, using physics-based simulation enriched the training far beyond what limited sensor data could.
We also learned the importance of contextual features. Including variables like outdoor temperature or special event flags significantly improved forecasts for buildings with temperature-sensitive loads or irregular usage. This matches prior observations that fine-tuning and feature choice must be made carefully for best performance.
On a more practical note, we confirmed that simplicity can win: often a well-pretrained general model outperforms a complex custom solution trained on sparse data. As one study notes, the ability of foundation models to achieve high accuracy with limited data and without retraining “opens up new possibilities for developing more efficient and accessible energy forecasting solutions”. We plan to publish these findings to help guide future research.
What's next for Short-term energy load forecasting
Looking ahead, we see several avenues to advance this work. One is probabilistic forecasting: extending the TSFM to output not just point forecasts but confidence intervals, which is important for risk-averse planning. Another is model adaptation: developing lightweight on-device versions of TSFMs so that forecasts can run locally at scale (e.g. one model serving hundreds of buildings via edge devices). We also aim to integrate control actions: coupling forecasts to optimization (e.g. scheduling HVAC or battery dispatch), thus moving from prediction to automated energy management.
On the research side, emerging models like Google’s TimesFM and others (Chronos, LagLlama) will continue evolving. We plan to explore fine-tuning strategies and federated learning so that private building data can improve forecasts without compromising privacy. Finally, because anomaly detection showed value, we will enhance that capability: for example, by training the TSFM jointly on forecasting and anomaly labels, or by using reconstruction-based scoring.
In summary, our TSFM-based approach lays a strong foundation: it delivers accurate, data-efficient forecasting and anomaly detection for building loads. With further refinements and integrations, it can play a key role in smarter energy systems that help buildings run cleaner and greener.
