BLACKBOOK

• 

  Decision Command Center — every IP with its PURSUE / WATCH / PASS recommendation, score, and the four component scores behind it.

• 

  Evidence Graph — every claim is a node carrying source URL, publication date, confidence and quote. Click a node to inspect its provenance.

• 

  Counterfactual stress test — the smallest change that flips PURSUE to WATCH, run through the same deterministic scoring engine.

• 

  Decision drift timeline — new evidence moved this IP from PURSUE to WATCH to PASS, each revision linked back to the one before it.

• 

  At the floor the engine says so: no swing moves this off PASS, so it reports margin erosion instead of inventing a false flip.

Inspiration

Film & TV development teams make million-dollar IP acquisition decisions based on fragmented research — scattered across trade publications, database lookups, and hallway conversations. When market conditions shift (a competitor announces a similar project, rights change hands, a reboot underperforms), there's no systematic way to detect that a previous "PURSUE" decision should be reassessed. We wanted to build a system that not only makes structured, evidence-backed IP decisions but remembers why and watches for changes.

What it does

BLACKBOOK is an AI-powered IP intelligence agent that:

1. Researches an IP — runs a parallel search loop (Parallel Search → Extract → Task) to gather evidence from the web.
2. Builds an evidence graph — every claim is a node/edge with source, timestamp, confidence, and relationship. No blob-of-text summaries.
3. Scores and recommends — Gemini evaluates four components (opportunity, competition, rights confidence, production feasibility), then a deterministic scoring function computes a final PURSUE / WATCH / PASS recommendation. The LLM explains; the code decides.
4. Persists decisions with memory — every decision, its evidence, and reasoning is stored with explicit lineage, forming a decision timeline.
5. Monitors for drift — a persistent Parallel Monitor tracks new events, deduplicates them, checks materiality, and triggers linked re-evaluations when material changes are detected.
6. Simulates counterfactuals — ask "what would have to change for you to recommend PASS?" and the system runs scenarios through the same deterministic scoring to find the smallest changes that flip the recommendation.

How we built it

Architecture: GitHub Webhook → FastAPI → LangGraph Agent → Slack Review → DataHub (MCP/REST), with PostgreSQL for event storage.

Agent pipeline: 8 LangGraph nodes — Parse → Classify → Detect → Generate → Score → Route → Approve → Writeback — orchestrated through a state machine.

Tech stack:

• Google Gemini (genai SDK + ADK) for LLM orchestration, evidence assessment, and structured outputs
• Parallel AI (Search, Extract, Task, Monitor) for the research loop
• Python / FastAPI backend with async SQLAlchemy
• React + TypeScript + Vite frontend (command center, evidence graph visualization, counterfactual explorer, timeline view)
• SQLite for decision memory and evidence persistence
• Docker + Google Cloud Run for deployment

Key design decisions:

• Deterministic scoring: Gemini produces component scores, but the final score and recommendation are computed from fixed weights in code. Identical inputs always produce identical outputs.
• Evidence graph over summary: every claim is traceable to its source with provenance.
• On-demand drift evaluation via POST /watch, with webhook-driven automatic re-evaluation as the next step.

Challenges we ran into

• LLM response parsing — ensuring reliable JSON extraction from Gemini outputs with fallback strategies for malformed responses.
• Async SQLAlchemy + greenlet issues — session management across async boundaries required the populate_existing query option to avoid stale data.
• Slack Block Kit complexity — designing interactive approval messages with action IDs and state tracking for the review workflow.
• Deterministic scoring vs. LLM creativity — keeping Gemini's explanatory role strictly separate from the scoring function so the system remains reproducible.
• Deduplication in drift detection — ensuring the same event never produces a repeated decision revision across monitor cycles.
• SQLite vs PostgreSQL compatibility — conditional engine configuration for pool parameters between local dev and production.

Accomplishments that we're proud of

• Full end-to-end pipeline: webhook → LLM analysis → evidence graph → deterministic scoring → decision memory → drift monitoring → linked re-evaluation.
• Counterfactual simulator that reuses the exact same scoring functions as the real engine — projected scores are guaranteed accurate.
• 43 passing tests with 0 lint violations, all offline (no API keys required).
• Production-grade CI/CD with Azure DevOps (build → test → push → staging → production).
• Thread-safe worker scheduler with graceful shutdown.
• LLM provider fallback with automatic retry.

What we learned

• Deterministic code + creative LLM is a powerful pattern: let the LLM explain and assess, let fixed code decide. This gives you both interpretability and reproducibility.
• Evidence graphs are far more useful than text summaries for decision-making — every claim is traceable, auditable, and comparable across time.
• Decision drift is a real, underserved problem. Most systems make a decision and forget about it. Monitoring for materiality changes adds enormous value for long-lived strategic decisions.

What's next

• Metadata auditor worker to proactively scan DataHub for schema drift (pull-based, not just event-driven).
• dbt integration for direct model-to-metadata lineage.
• Multi-repository support with org-wide webhook management.
• Dashboard with proposal metrics and approval workflow analytics.
• GitHub PR comments showing metadata impact alongside code changes.
• DataHub Business Glossary and Tag propagation for compliance workflows.

Built With

• development
• docker
• fastapi
• google-cloud
• google-gemini
• kit
• parallel-ai
• playwright
• pydantic
• python
• react
• sqlite
• typescript
• vite