MorningPulse: Autonomous Telemetry Intelligence

Inspiration

Modern enterprises suffer from dashboard fatigue. While engineering, finance, growth, and customer support teams generate millions of telemetry records daily, leadership often learns of critical regressions—like a failing checkout gateway or an escalating customer churn rate—hours after revenue has already leaked.

Existing Business Intelligence (BI) tools are largely passive, requiring executives to sift through dozens of charts across multiple tabs. We were inspired to build MorningPulse: an autonomous AI Chief of Staff that runs natively within enterprise data warehouses, continuously parses operational telemetry, detects statistical anomalies overnight, and delivers a concise, actionable executive briefing directly to inboxes by 8:00 AM.

---

How We Built It

MorningPulse is engineered with a modular, security-first architecture designed to operate entirely inside the enterprise data boundary.

1. In-Warehouse Data Layer & Schema Mapping

We ingested 14 days of cross-departmental telemetry across Finance, Engineering, Product, and Support into Snowflake. To accommodate diverse corporate naming conventions, our ingestion engine (core/detector.py) applies dynamic schema resolution to normalize various telemetry representations into standard vectors:

$$\text{Telemetry Point} = \langle t_k, \, m_i, \, v_{i,k}, \, d_i, \, u_i \rangle$$

where:

• t_k ∈ ℝ represents the reporting timestamp / date
• m_i ∈ M denotes the telemetry metric identifier
• v_{i,k} ∈ ℝ is the recorded observation value
• d_i ∈ {Finance, Engineering, Product, Support} designates the domain department
• u_i is the unit of measure (e.g., USD, %, ms)

2. Statistical Anomaly Detection Engine

Rather than relying on brittle, static threshold alerts, MorningPulse models baseline distributions dynamically using a rolling window of historic observations W = {v_{i, 1}, v_{i, 2}, ..., v_{i, N}} (where N = 13 days prior to the current target observation v_{i, today}).

For each metric m_i, we compute the empirical baseline mean μ_i and sample standard deviation σ_i:

$$\mu_i = \frac{1}{N} \sum_{k=1}^{N} v_{i, k}$$

$$\sigma_i = \sqrt{\frac{1}{N - 1} \sum_{k=1}^{N} (v_{i, k} - \mu_i)^2}$$

The current day's normalized deviation is evaluated using the standard Z-score:

$$Z_i = \frac{v_{i, \text{today}} - \mu_i}{\sigma_i}$$

An observation is flagged as an anomaly if |Z_i| ≥ Z_crit (where Z_crit = 1.96, corresponding to a 95% two-tailed confidence interval under normal assumptions). Anomalies are automatically bifurcated into Risks (R) and Opportunities (O) using directional polarity functions:

$$\mathcal{R} = { m_i \mid (Z_i \ge Z_{\text{crit}} \land \text{dir}(m_i) = \text{lower}) \lor (Z_i \le -Z_{\text{crit}} \land \text{dir}(m_i) = \text{higher}) }$$

$$\mathcal{O} = { m_i \mid (Z_i \ge Z_{\text{crit}} \land \text{dir}(m_i) = \text{higher}) \lor (Z_i \le -Z_{\text{crit}} \land \text{dir}(m_i) = \text{lower}) }$$

3. In-Situ Cortex Generative Synthesis

Identified outlier vectors are structured and routed to Snowflake Cortex via the AI_COMPLETE SQL interface, utilizing hosted foundational models such as llama3.1-70b and snowflake-arctic. The model synthesizes raw mathematical deviations into:

• The Bottom Line: High-level impact on quarterly OKRs and top-line margins.
• Top Operational Risks: Root-cause hypotheses and affected infrastructure components.
• Growth Windfalls: Emerging opportunities to capitalize on immediately.
• Directives & Remediation: Concrete triage steps for department heads.

4. Interactive Chief of Staff & Executive UI

The application is deployed on Streamlit, featuring:

• Executive Morning Pulse: High-level alert cards and staged one-click remediation actions.
• Company Cockpit: Multi-metric Plotly telemetry trends with status color tagging.
• Mailing List Directory: Secure recipient management with masked privacy controls.
• Live Cortex Copilot: A sidebar conversational agent allowing executives to drill into anomalies in real time.

---

Challenges We Faced

1. SQL REST API v2 Serialization: Authenticating and querying Snowflake through the v2 REST API required handling asynchronous query polling (202 Accepted response handling) and stripping quotes across database/schema identifiers to prevent casing mismatches.
2. Zero Data Egress Compliance: Ensuring enterprise privacy meant raw telemetry could not be forwarded to third-party public LLM endpoints. We solved this by executing generative inference directly inside Snowflake's security perimeter with Cortex AI_COMPLETE.
3. Environment Parity & Deployment: Transitioning from a local .env setup to cloud deployment required designing robust fallback configuration logic between os.environ and Streamlit's TOML-based st.secrets.
4. False Positive Suppression: In metric datasets with minimal variance, raw standard deviations approach zero (σ_i → 0), which causes standard scores to blow up. We stabilized the detection threshold by implementing an absolute percentage delta floor:

|Δ%i| = |(v{i, today} - μ_i) / μ_i| × 100% ≥ 15%

---

What We Learned

• In-Database Generative AI is the Future: Running LLMs directly where the data resides completely removes compliance hurdles, data pipelines, and egress latency.
• Statistical Rigor Beats Unstructured Prompts: LLMs are most effective when fed mathematically pre-filtered signals. Combining deterministic Z-score detection with probabilistic generative synthesis yields reliable executive insights without hallucinations.
• Designing for Executive Velocity: The best developer tools and analytics platforms do not overwhelm users with more charts; they minimize time-to-insight and accelerate executive decision-making.

Built With

• api
• cortexapi
• gemini
• pandas
• python
• request
• snowflake
• snowflakedatabase
• sql
• streamlit