LoopLens

LoopLens helps engineering teams prove whether AI coding tools are actually improving delivery by connecting AI activity to validation, reviews, CI, deployments, and production outcomes.

Comment

Inspiration

AI coding tools have rapidly changed how software is built. Engineers can now generate code, investigate failures, and iterate much faster with tools like Codex, Claude Code, and Cursor.

However, engineering leaders still lack a reliable way to answer a more important question:

Is AI actually helping the organization deliver production-ready software faster, or is it simply producing more code, more retries, and more review rework?

Most existing analytics products focus on activity: prompts sent, tokens consumed, lines changed, commits created, or estimated time saved. These signals are useful, but they do not prove that the resulting work was correct, reviewable, or successfully delivered.

That led us to build LoopLens, an evidence layer for AI-assisted engineering.

LoopLens is deliberately not a developer-ranking product. It does not try to decide whether one engineer is better than another. Instead, it measures whether the surrounding delivery system converts AI-assisted effort into verified, production-ready outcomes.

What it does

LoopLens connects AI development activity with the systems where engineering outcomes become visible: local validation, GitHub, CI, deployment infrastructure, and Jira.

A privacy-safe local collector observes structured workflow events from Codex, Claude Code, and Cursor. It captures signals such as:

  • Session and attempt boundaries
  • Tool execution outcomes
  • File-change events
  • Git branches and commit identifiers
  • Required validation results
  • Validation duration
  • Code-state fingerprints
  • Token counts, source, and confidence
  • Collector health

The collector sanitizes information before it enters its local queue or reaches the network. LoopLens never needs to upload source code, raw prompts, AI responses, shell output, stack traces, environment variables, credentials, or transcripts.

LoopLens connects these coding events to delivery evidence from Jira and GitHub:

  • Jira work items and workflow history
  • Planned task complexity
  • Git branches and commits
  • Pull requests
  • Review approvals and change requests
  • CI checks and workflow conclusions
  • Merge timestamps
  • Production deployments
  • Reopen, revert, and other quality signals

LoopLens provides views for engineering-system health, work items, AI coding sessions, tools and models, review outcomes, integrations, and evidence-grounded AI recommendations.

A leader can ask:

What should this team improve this week?

Instead of returning generic management advice, LoopLens Copilot receives a bounded, tenant-specific evidence pack and returns specific actions, supporting metrics, data sources, and confidence.

How we built it

LoopLens is organized as a TypeScript monorepo with separate packages for the web application, collector CLI, database schema, shared event contracts, and metrics engine.

Local collection

The LoopLens CLI installs provider-aware hooks for Codex, Claude Code, and Cursor.

Hook payloads are converted into a common event schema containing:

  • Repository
  • Worktree
  • Branch
  • HEAD SHA
  • Code state
  • Provider conversation
  • Model context

Raw information is transformed locally before transmission:

  • Prompt text becomes structural features and a hash
  • Failure output becomes a normalized HMAC-SHA256 fingerprint
  • Absolute paths become repository-relative identifiers
  • Validation commands become configured command IDs and outcomes
  • Token measurements include their source and confidence

A strict privacy contract rejects forbidden fields before an event can be queued or transmitted.

Events are written to a local SQLite spool. A background daemon retries delivery and uploads batches to the Vercel ingestion API. Device credentials authenticate the collector, and event IDs make ingestion idempotent.

Vercel and Next.js

The application frontend and backend route handlers run as a Next.js application deployed on Vercel.

The backend includes:

  • Collector bootstrap and device-token exchange
  • Batch event ingestion
  • Dashboard data APIs
  • GitHub webhook handling
  • Jira OAuth callbacks and webhooks
  • Team and identity management
  • Work-item complexity APIs
  • Copilot chat
  • Scheduled connector synchronization
  • Inactive-session finalization

WorkOS AuthKit provides authentication, organization context, and role-based access control. Tenant context is applied to application queries so each organization only accesses its own evidence.

GitHub and Jira

The GitHub App receives signed webhook events. LoopLens verifies the webhook signature before processing installation, repository, pull-request, review, CI, and deployment events.

Jira is connected through OAuth 2.0. Refresh tokens are encrypted using AES-256-GCM before storage.

Jira projects, issue state, workflow history, and complexity settings are normalized into the same delivery model used for GitHub and collector events.

Webhook payloads are normalized in request memory. Raw Jira and GitHub webhook bodies are not retained.

Evidence engine

Collector, Jira, and GitHub events are converted into immutable normalized lifecycle events.

LoopLens then derives rebuildable projections for:

  • Provider sessions
  • Branch and worktree segments
  • Local-green state
  • Session efficiency metrics
  • Work-item attribution
  • Planned and observed complexity
  • Pull-request outcomes
  • Cycle-time stages
  • Quality signals
  • Portfolio metrics
  • Copilot recommendations

Separating immutable evidence from derived projections means metrics can be recalculated when attribution logic, workflow mappings, or complexity rules change.

AWS Aurora PostgreSQL

AWS Aurora PostgreSQL is the primary backend database.

It stores:

  • Organizations, users, memberships, and roles
  • Projects, devices, and teams
  • Provider sessions and AI workflow events
  • Validation runs and session metrics
  • Collector health
  • GitHub installations and repositories
  • Pull requests, reviews, CI runs, and deployments
  • Jira connections, projects, work items, and status history
  • Normalized lifecycle events
  • Work segments and attribution links
  • Cycle-time projections
  • Complexity inputs, metrics, and overrides
  • Recommendations and model pricing

The application uses Drizzle ORM and PostgreSQL pooling. Production connections use verified TLS and support Vercel-to-AWS IAM/OIDC authentication through the AWS RDS signer.

Challenges we ran into

Normalizing different AI tools

Codex, Claude Code, and Cursor expose different hooks, identifiers, token information, and session behavior. Treating them as identical would have produced misleading metrics.

We created a shared event contract while preserving provider-specific token sources and confidence levels.

Defining a trustworthy local green

A passing test is not meaningful if the code changes immediately afterward. We needed to associate validation results with a specific code state and invalidate previous results after meaningful edits.

This made local green an evidence-backed state rather than a simple “last command passed” flag.

Connecting coding sessions to business work

AI conversations, branches, commits, Jira issues, and pull requests do not share one universal identifier.

We built a work graph using Jira keys, repository context, branch and worktree segments, commits, pull requests, and explicit attribution confidence. The system displays uncertainty rather than silently claiming perfect attribution.

Maintaining privacy without destroying usefulness

Many useful developer analytics implementations collect prompts, transcripts, diffs, or shell output. That would make LoopLens difficult to trust and difficult to deploy.

We designed the event model around derived signals, counts, hashes, fingerprints, outcomes, and identifiers. Privacy enforcement exists both locally and at API ingestion.

Handling asynchronous and duplicate events

Collector events, GitHub webhooks, Jira updates, and scheduled synchronization can arrive late, out of order, or more than once.

We used event identifiers, delivery IDs, idempotency keys, database uniqueness constraints, transactions, and rebuildable projections to make the system resilient.

Producing useful AI recommendations

It is easy to add a chatbot that produces plausible but generic advice. It is much harder to make the output operationally useful.

We built a bounded evidence pack from the organization’s metrics and required the Copilot response to include concrete actions, supporting evidence, and confidence.

Accomplishments that we're proud of

We built a working full-stack B2B product rather than a static analytics mockup.

The system includes:

  • A real local collector and durable SQLite queue
  • Multi-provider AI workflow normalization
  • Privacy-safe event contracts
  • Authenticated batch ingestion
  • WorkOS authentication and organization RBAC
  • A production Next.js application deployed on Vercel
  • AWS Aurora PostgreSQL as the primary database
  • A functioning GitHub App and signed webhook processing
  • Jira OAuth, synchronization, and workflow mapping
  • A normalized work graph
  • Strict code-state-aware local-green measurement
  • Planned versus observed complexity
  • Review, CI, deployment, reopen, and revert outcomes
  • Evidence-grounded AI recommendations
  • Unit and end-to-end tests

We are especially proud that LoopLens avoids reducing engineering performance to one simplistic score. It preserves the evidence and confidence behind each conclusion.

What we learned

We learned that AI engineering analytics becomes valuable only when it is connected to delivery outcomes.

Prompt counts and token usage are not enough. The strongest signals come from the relationships between:

  • AI attempts
  • Code-state-aware validation
  • CI results
  • Human review
  • Jira workflow progress
  • Production deployment
  • Post-delivery quality

We also learned that confidence must be treated as first-class data. Token measurements, work-item attribution, complexity estimates, and incomplete lifecycle events all have different certainty levels. Exposing that uncertainty makes the product more credible.

Privacy architecture cannot be added at the end. The event model itself must be designed so the server never needs sensitive raw content.

Finally, AI recommendations improve significantly when the model is given a small, structured evidence pack instead of a large collection of raw engineering data.

What's next for LoopLens

The next step is turning LoopLens from a strong working product into a production-scale engineering intelligence platform.

Planned improvements include:

  • Additional CI and deployment integrations
  • Deeper GitHub repository synchronization
  • More robust Jira onboarding and webhook renewal
  • GitLab, Linear, Azure DevOps, and Bitbucket support
  • Configurable delivery-stage definitions
  • Team-specific baselines and trend analysis
  • Automated weekly engineering-system reports
  • Proactive alerts for retry loops, CI disagreement, and review bottlenecks
  • Recommendation follow-up and impact measurement
  • Organization-level retention and governance controls
  • SAML, SCIM, audit logs, and enterprise policy management
  • More granular cost attribution across providers and models
  • Controlled experiments comparing AI-assisted and non-AI workflows
  • Rebuildable historical analysis when mappings or attribution rules change

Our long-term goal is for LoopLens to become the evidence layer engineering organizations use to improve AI-assisted delivery without monitoring or ranking individual developers.

Built With

  • aes-256-gcm-connector-token-encryption
  • aws-aurora-postgresql
  • aws-sdk-rds-signer
  • better-sqlite3
  • claude-code-hooks
  • codex-hooks
  • commander
  • css
  • cursor-workflow-collection
  • drizzle-kit
  • drizzle-orm
  • durable
  • event
  • github-app
  • github-webhooks
  • gpt-5-mini
  • hmac-sha256-failure-fingerprints
  • html
  • iam/oidc-database-authentication
  • javascript
  • jira-cloud-rest-api
  • jira-oauth-2.0
  • jira-webhooks
  • jose
  • js-tiktoken
  • jwt-verification
  • local
  • lucide-react
  • next.js
  • next.js-route-handlers
  • node-postgres
  • node.js
  • octokit-webhook-verification
  • openai-api
  • organization-based-multitenancy
  • pnpm-workspaces
  • provider-hooks
  • react
  • recharts
  • role-based-access-control
  • shell-hooks
  • shell-scripting
  • signed-github-webhook-verification
  • sql
  • sqlite
  • strict-privacy-schemas
  • tailwind-css
  • tsup
  • typescript
  • vercel
  • vercel-cron
  • vercel-functions-database-pooling
  • vercel-serverless-functions
  • verified-tls-database-connections
  • workos-authkit
  • zod
Share this project:

Updates