AgentOps Debugger — OEFA Environmental Compliance

Auditable environmental compliance agents — ask about OEFA (Peru) sanctions in plain Spanish or English and get evidence-cited answers and reports from Qwen agents that show their work

Comment

Inspiration

Environmental compliance work depends on scattered public records, legal documents, sanctions data, and procedural history. Analysts need answers they can defend, not just chatbot summaries. That inspired AgentOps Debugger — OEFA Environmental Compliance: an agentic system that helps investigate regulated entities using OEFA public data while making every conclusion traceable to evidence. The name is the thesis — the agent's work is debuggable: every answer ships with the trace that produced it.

What it does

AgentOps Debugger lets users ask natural-language questions — in Spanish or English — about environmental compliance history. It retrieves OEFA records and regulatory documents, builds cited answers, generates charts, drafts structured reports behind a human approval gate, and shows the full agent trace behind the result. You can also start without knowing any entity: ask for a listing of sanctioned companies and click one to launch the investigation, with state-aware suggestions guiding each next step.

Instead of behaving like a black box, AgentOps Debugger exposes:

  • the agent plan
  • task execution
  • retrieved evidence
  • generated charts
  • report drafts
  • human approval steps
  • guardrail decisions
  • final citations

How we built it

We built the project as a TypeScript monorepo with:

  • React + Vite for the frontend workspace
  • Hono + Node.js for the backend API
  • zod shared contracts between frontend, backend, and orchestration
  • Qwen on Qwen Cloud (DashScope, OpenAI-compatible) via Mastra + AI SDK v5 for the planner and specialist agents
  • offline fallback agents so the app works without API keys
  • RAG retrieval over seeded regulatory documents
  • OEFA data normalization from public records
  • evidence guardrails to drop unsupported claims
  • HITL approval before saving reports
  • PDF/DOCX/XLSX export for generated reports
  • Tablestore and OSS clients behind storage ports (durable sessions, ledger, and report files; in-memory offline)
  • a single Docker container (API + SPA) deployed on Alibaba Cloud ECS, with edge hardening (rate limiting, structured logging with redaction, deep health checks)
  • full ES/EN internationalization, including translated citations with a "show original" toggle

Challenges

The hardest part was making the system auditable. It is not enough for an agent to answer correctly; it must show why the answer is justified. That required a typed event stream, a persistent ledger, citation contracts, and guardrails that reject findings without evidence.

Another challenge was handling messy real-world public data. Field names, formats, dates, entity names, and legal statuses can vary. We had to normalize records, preserve source context, and avoid guessing when entity matches were ambiguous.

Deploying against the real model taught us that offline tests structurally can't catch everything: live Qwen output varies in ways strict schemas reject — improvised status labels, missing fields, English labels in Spanish contracts. We fixed four rounds of live-only bugs with tolerant schemas that coerce and salvage instead of failing, plus prompt contracts that pin the expected shapes.

We also wanted the project to be demoable without credentials, so we built an offline mode with seed data and deterministic agents while keeping the live Qwen/OEFA path available when configured.

What we learned

We learned that useful agentic applications need more than prompts. They need contracts, state, evidence, validation, observability, and human control.

The biggest lesson was:

An agent system becomes trustworthy when every conclusion can be traced back to the data, documents, and decisions that produced it.

What's next

Next steps: authentication and authorization, live integration with the full OEFA dataset at scale, Stage-2 cloud state (Tablestore/OSS in the deployed environment), deployment automation, and a richer evaluation set for citation accuracy.

Built With

  • alibaba-cloud-oss
  • alibaba-cloud-tablestore
  • bm25
  • claude
  • codex
  • dashscope
  • docx
  • exceljs
  • hono
  • junar-api
  • mastra
  • node.js
  • oefa-datos-abiertos
  • pdfkit
  • pnpm-workspaces
  • qwen-cloud
  • rag
  • react
  • recharts
  • tailwind-css
  • tanstack-query
  • tanstack-router
  • typescript
  • vercel-ai-sdk
  • vite
  • vitest
  • zod
Share this project:

Updates