Sigma Skibidi

Inspiration

Buildings account for a large share of energy use and emissions, yet most owners only find out how efficient a building is after the utility bills arrive. We wanted to know whether a building's energy use could be predicted ahead of time from things that are already known: what the building is, how big and how old it is, and what the weather was like. If that works, it becomes much easier to spot buildings that use far more energy than they should and target them for retrofits.

The WiDS Datathon 2022 data gave us a realistic version of this problem. It has about 76,000 records of real buildings across several anonymised US states and seven years, and it asks for a prediction of site_eui (Site Energy Usage Intensity: the energy a building uses per square foot per year). We also wanted to see how far we could get using only MATLAB and the models in its Regression Learner app.

What it does

Sigma Skibidi is a MATLAB pipeline that predicts site_eui for 9,705 buildings in a future year (year 7) that the model has never seen. It learns from 75,757 training records covering years 1 to 6.

The final model uses 66 predictors:

• 57 original columns: building class, facility type, floor area, year built, Energy Star rating, elevation and the local weather for that year (monthly temperatures, heating and cooling degree days, rain, snow, and days above or below temperature thresholds).
• 5 engineered features: log_floor_area, winter_avg_temp, summer_avg_temp, annual_temp_range and an energy_star_missing flag.
• 4 building history features: the building's own Site EUI in earlier years (lag_last, lag_mean, lag_count, lag_gap).

The prediction is the average of two Regression Learner models, Bagged Trees and tuned Boosted Trees. On a validation set built to look like the test set, it has an RMSE of 25.99, compared with 64.88 for always predicting the average and 46.84 for our best model before we added building history. Running runAll in MATLAB reproduces everything from raw CSVs to submission.csv.

How we built it

1. Explored what makes the test set different. The test set is a single future year, so Year_Factor cannot be a useful predictor and the model has to generalise forward in time. State_6 makes up 67% of the training data but does not appear in the test set at all. The wind and fog columns are 88 to 94% missing in the test set, so we dropped them. About 35% of Energy Star ratings are missing, and some year_built values are 0 placeholders.

2. Built a validation split that copies the test set. Our first attempt used five-fold cross-validation, but random folds let the model see the same year it is scored on, which gives over-optimistic scores. We switched to training on years 1 to 5 and validating on year 6, and we score that year twice: on all states and "test-like" (9,432 rows without State_6). We chose models on the test-like score, then retrained the winner on all six years for the submission.

3. Cleaned the data and engineered features. We treated year_built below 1600 as missing, took the log of floor area because it spans several orders of magnitude, summarised the 36 monthly temperature columns into seasonal averages and a range, and flagged buildings with no Energy Star rating, because unrated buildings behave differently.

4. Added building history, our biggest improvement. The data has no building id, but a building keeps the same state, class, facility type, floor area and year built every year. We used that combination as a key to match each building across years, then gave every row the building's Site EUI from strictly earlier years only, so no information leaks from the year being predicted. 93.4% of test rows have history in the training data. This took our best test-like RMSE from 46.84 to 25.99.

5. Fixed how trees handle missing values. We found that MATLAB trees without surrogate splits stop an observation with a missing value at the split, instead of sending it down a branch. We wrote fillSentinel, which replaces missing numbers with -999 (a value no real input takes) so "missing" can get its own branch. The linear baseline uses median imputation instead.

6. Compared models and picked from the evidence. We trained the mean baseline, Linear, Fine Tree, Medium Tree, Neural Network, Bagged Trees and Boosted Trees, including Boosted Trees on a log target. We tuned Boosted Trees over a small grid (learning rate 0.1, 127 or 255 splits, leaf size 10 or 30) and chose the number of trees from the validation curve. Averaging Bagged Trees and tuned Boosted Trees beat either one alone, because the two ensembles make different errors that partly cancel.

Challenges we ran into

• Choosing a validation that we could trust. The five-fold cross-validation in our first draft gave scores that looked good but did not reflect the real task of predicting a new year. Moving to a year-based split, and then removing State_6 from the score, made our numbers worse on paper but much more honest.
• The test set is not like the training set. One state dominates training and is missing from the test, and several columns are almost empty in the test. We had to decide what to drop and what to score against before any modelling.
• Finding the signal. With around 60 raw predictors, it was not obvious what mattered. Weather and building size gave only moderate gains, and our tree models plateaued at an RMSE around 47. The breakthrough came from noticing that buildings repeat across years even without an id column.
• Avoiding leakage. Building history is powerful but easy to get wrong. If a row could see its own year's value, validation would look perfect and the submission would fail. We built the history features year by year using only earlier years.
• Missing values in MATLAB trees. Predictions for rows with missing values were quietly wrong until we traced it to how trees treat NaN at prediction time without surrogate splits.
• Time. We had about three hours to go from raw data to a submission, which pushed us to keep the pipeline simple, scripted and reproducible.

Accomplishments that we're proud of

• Cutting the test-like RMSE from 64.88 (predicting the average) and 46.84 (best model without history) to 25.99, a 60% reduction from the baseline.
• Finding the building history idea ourselves from the data. It turned out to be the same core idea the first-place WiDS 2022 team built on.
• A validation design that mirrors the real test set, so we can trust the number we report.
• A clean, reproducible MATLAB pipeline: three numbered scripts and one runAll command that goes from raw CSVs to a checked submission.csv (9,705 rows, all finite, all at least 1).
• Using only models available in Regression Learner, so every result can be reproduced and explored in the app.

What we learned

• A building's own past is the best predictor of its future. In the final models, lag_last alone accounts for 27 to 42% of predictor importance, and facility type for another 18 to 19%. Weather adds comparatively little once history is known.
• Validation design matters as much as the model. A split that does not match the test set can make a weak model look strong. Thinking about how the test set differs from the training data shaped every later decision.
• Feature engineering beat model choice. Swapping models moved the RMSE by a few points. Adding four history features cut it almost in half for every model, even the linear one (54.26 to 35.83).
• Check usefulness with evidence, not names. Several predictors that sound important, such as detailed monthly temperatures, mattered less than expected, and a missing-value flag mattered more. Predictor importance and validation scores told us what to keep.
• Know your tools' defaults. How MATLAB trees handle missing values changed our predictions, and we only found it by checking.
• Simple ensembles help. Averaging two different tree ensembles gave a small but reliable gain at almost no cost.

Results

• Code and submission: https://github.com/dgmtriet/Hackathon2026 (branch lam_workings)
• Final validation score: test-like RMSE 25.99 on year 6 (excluding State_6)
• Submission: submission.csv, 9,705 predictions in id,site_eui format (mean 72.4, median 61.2)

What's next for Sigma Skibidi

• Separate models for buildings with and without history. The 6.6% of test buildings with no past record probably need a different model than the rest.
• Out-of-year group averages, such as average Site EUI per facility type or per state, computed only from other years, to help the buildings without history.
• More tuning and a larger ensemble, such as more trees, different leaf sizes and more models in the average.
• Better building matching. Some buildings may change floor area or have missing build years, which breaks the current key. A fuzzier match could recover more history.
• Explain the predictions to building owners, for example by flagging buildings whose actual use is far above the model's prediction as candidates for an energy audit.

Built With

• claude
• matlab