CloudOps_TehAis

Secure & Energy-Aware Cloud Platforms

Comment

Inspiration

HILTI's actual cloud stack (AWS + EKS, provisioned through Crossplane and synced via Flux GitOps) is managed across multiple departments with very different levels of cloud expertise, exactly the setup where misconfigurations and wasted resources creep in unnoticed. Digging into HILTI's own 2025 sustainability report sharpened the pitch further: HILTI reports zero market-based Scope 2 emissions, but that's achieved through renewable energy certificates covering their own buildings and manufacturing, not their cloud vendor's data centres. Cloud compute emissions sit in Scope 3, a category their existing 100% green electricity program doesn't touch. That's a real, citable, currently-unaddressed gap, and it's what CloudOps targets.

What it does

CloudOps is a manager-friendly dashboard that combines three things HILTI's cloud team needs but doesn't have in one place:

i) Security configuration scanning - flags real misconfiguration patterns (public storage, exposed server access, excessive IAM permissions, insecure public endpoints, over-permissive serverless roles) against professional security baselines

ii) ML-driven workload optimisation - forecasts near-term demand, detects idle and over-provisioned VMs, and recommends concrete actions: schedule shutdown, downsize, raise or lower autoscaling bounds, or investigate an anomaly

iii) Carbon, energy, and cost scoring - converts every recommendation into estimated kWh saved, CO2e avoided, and dollar savings, using published methodology rather than invented numbers

Everything surfaces through one Streamlit dashboard with separate views for managers (plain-language priorities) and technical teams (the underlying detail).

How we built it

i) Trained three models - an XGBoost demand forecaster, an XGBoost rightsizing classifier, and an Isolation Forest anomaly detector, on real Azure Public Dataset V2 VM traces, used as a provider-agnostic stand-in since no public HILTI cloud telemetry exists. CPU utilisation behaviour (idle, over-provisioned, spiking) is fundamentally provider-agnostic, so patterns learned here transfer meaningfully to an AWS/EKS environment.

ii) Verified the dataset's actual raw schema against Azure's published schema.csv

iii) Grounded the idle-detection rule in AWS Trusted Advisor's real published low-utilisation threshold (≤10% average CPU) instead of inventing one.

iv) Estimated energy and carbon impact using the actual Cloud Carbon Footprint methodology and its published AWS power coefficients, converted to CO2e using GHG-Protocol-style accounting.

v) Read through HILTI's real 2025 annual report to ground the business case in disclosed facts: an 8% year-over-year emissions reduction, SBTi-validated targets, and the Scope 2/Scope 3 distinction above.

vi) Built a lightweight Python backend that ties the ML output, the carbon calculator, and the security scanner together, feeding a single Streamlit frontend.

vii) Handed the ML output schema to teammates as a documented contract (with sourcing, methodology, and known limitations spelt out) so the carbon-scoring and dashboard work could be built independently and still integrate cleanly.

Challenges we ran into

i) No public HILTI cloud telemetry exists at all, so the first real challenge was finding and justifying a credible substitute dataset rather than simulating one outright.

ii) Processing the dataset hit real memory and I/O limits (an out-of-memory crash on the full join, intermittent read timeouts on compressed files in our sandboxed environment), the loading pipeline had to be redesigned around streaming and sampling instead of loading everything at once.

iii) Keeping multiple real frameworks (Cloud Carbon Footprint, GHG Protocol, SBTi, AWS Trusted Advisor) cleanly separated, so no single number ever quietly borrowed credibility from a standard it wasn't actually using.

Accomplishments that we're proud of

i) Every number in the pipeline traces back to a real, citable source, no invented carbon factors, no made-up thresholds.

ii) Three independently-built modules - security scanning, ML workload optimisation, carbon scoring, integrate end-to-end into one working dashboard, not three disconnected demos.

What we learned

The real difference between an ML prediction, a rule-based threshold, and a framework formula, and why blending them without being explicit about which is which risks overclaiming what the system actually does.

What's next for CloudOps_TehAis

i) Replace the Azure dataset stand-in with real HILTI telemetry once it's available.

ii) Extend the demand forecast horizon from the current ~3 hours to a true 24-hour window with more historical data.

iii) Connect recommendations to HILTI's actual GitOps tooling (Karpenter, Flux) so they can be applied with review and approval, not just read off a dashboard.

iv) Add a quantitative SBTi target-progress view once HILTI's specific numeric reduction target is publicly available to calculate against.

Built With

Share this project:

Updates