FedFathom: Agentic Market Memory

FedFathom helps small federal contractors turn award data into evidence-backed market briefs that remember only approved research preferences—making every analysis more focused and transparent.

Comment

What it does

Small federal contractors must decide where to spend limited business-development time, but federal award data is enormous, fragmented, and difficult to turn into a focused market strategy.

FedFathom: Agentic Market Memory turns a completed federal market Search into a five-section Capability-to-Market Fit Brief:

  1. Fit summary
  2. Agencies and offices buying similar work
  3. NAICS and PSC classifications
  4. Vendors winning related awards
  5. Possible upcoming market windows

Each brief includes deterministic metrics, analyzed scope, methodology, caveats, representative awards, and links to official USAspending evidence.

The central experience is a visible, user-controlled memory loop:

  1. FedFathom creates Version 1 from an immutable Search Execution.
  2. The user provides guidance such as, "Exclude staffing-heavy work and prioritize civilian health agencies."
  3. Amazon Bedrock converts only that user-authored instruction into an allowlisted structured proposal.
  4. FedFathom shows the proposal, its scope, and its expected effect before saving anything.
  5. After the user confirms it, CockroachDB stores the preference, its approval and lifecycle state, and its vector embedding.
  6. Version 2 retrieves applicable confirmed memories and reanalyzes the same immutable evidence.
  7. The comparison shows which memory was applied, how many records it excluded, and how the resulting market perspective changed. Version 1 remains unchanged.

This is not hidden chat personalization. The memory is explicit, inspectable, scoped, supersedable, and deletable.

The model cannot independently search for awards, select evidence, calculate metrics, access credentials, or write memory without confirmation. FedFathom controls evidence, authorization, calculations, and citations; Bedrock performs bounded synthesis.

How we built it

FedFathom uses React and TypeScript for the web experience and an Express API for authenticated orchestration. Existing PostgreSQL data remains authoritative for customer identities, subscriptions, Search Executions, and federal award evidence.

CockroachDB is the durable system of record for the new agent lifecycle. It stores:

  • Confirmed structured research memories and 512-dimensional embeddings
  • Approval, trust, scope, supersession, and deletion state
  • Account-scoped idempotency and report allowances
  • Work queues, renewable leases, fencing tokens, checkpoints, and retries
  • Immutable report versions, citations, provenance, and bounded events

We meaningfully integrated three CockroachDB tools:

  • Distributed Vector Indexing retrieves relevant memories after hard account, profile, approval, trust, deletion, and supersession filters. Similarity never replaces authorization.
  • CockroachDB Cloud Managed MCP Server powers a read-only Memory Inspector over a dedicated synthetic-only cluster. It demonstrates confirmed-memory retrieval and report, checkpoint, and recovery lineage without becoming part of the customer request path.
  • ccloud CLI supports repeatable CockroachDB Cloud provisioning and operational verification.

An AWS ECS Fargate worker claims report jobs from CockroachDB, obtains a deterministic evidence bundle from FedFathom, and invokes allowlisted Amazon Bedrock models without tools or database access. It validates returned citations and calculated values before committing an immutable report version.

Security boundaries include forced row-level security, server-derived account identity, purpose-specific database roles, composite tenant relationships, bounded telemetry, and source-controlled migrations.

FedFathom existed before the hackathon as a proprietary React, Express, and PostgreSQL federal market-research product. The submitted project is the newly created, open-source Agentic Market Memory engine and synthetic reference application.

Hackathon-period work includes confirmation-gated structured memory, CockroachDB lifecycle state and vector retrieval, bounded Bedrock synthesis and embeddings, ECS lease and checkpoint recovery, the synthetic reference journey, security and benchmark evidence, and the public release package. The repository documents this boundary in detail.

Challenges we ran into

The hardest challenge was making memory useful without turning it into an unsafe transcript store. Public award text, retrieved content, and model output must never silently become trusted long-term memory.

Our solution was to accept memory candidates only from user-authored instructions, translate them into an allow-listed schema, and require explicit confirmation before persistence. Retrieved content can inform a report, but it cannot write policy.

A second challenge was coordinating two authoritative databases. PostgreSQL owns existing customer and federal-evidence domains, while CockroachDB owns memory and report orchestration. We designed explicit ownership boundaries, idempotent requests, atomic allowance reservations, immutable versions, and renewable leases rather than attempting a risky database migration.

Tenant isolation required more than adding an account_id filter. We combined server-derived identity, forced row-level security, purpose-specific roles, composite tenant relationships, adversarial cross-account tests, and uniform not-found responses.

We also discovered that a read-only Managed MCP connection exposed a broader metadata surface than our intended two-view demonstration boundary. Instead of assuming that "read-only" was sufficient, we created a dedicated synthetic-only CockroachDB cluster for MCP inspection. This reinforced an important lesson: agent tools must be validated through observed behavior, not trusted solely because of their labels.

Finally, interrupted model work had to be recoverable without creating duplicate reports or consuming a user's allowance twice. CockroachDB-backed leases, fencing, checkpoints, idempotency, and immutable version commits make worker termination a tested state transition rather than an exceptional mystery.

Accomplishments that we're proud of

We are most proud that CockroachDB is not merely a vector-search sidecar. It is the durable memory and coordination layer for the complete agent lifecycle: confirmed context, embeddings, transactions, allowances, execution state, recovery, versions, provenance, and deletion.

Memory also produces a visible and testable effect. A user can compare report versions, inspect the exact preference that was applied, and see the resulting excluded-record count. Personalization is not hidden inside a prompt.

We built the system around real production concerns:

  • Explicit consent before memory persistence
  • Distributed-vector retrieval with tenant and trust filters
  • Exactly-once report-version and allowance behavior
  • Worker interruption and checkpoint recovery
  • Forced row-level security and adversarial cross-tenant denial
  • Deterministic calculations and evidence-linked synthesis
  • Citation and calculated-value validation
  • Telemetry that excludes prompts and sensitive content
  • A read-only MCP demonstration isolated from customer data
  • Reproducible open-source setup with synthetic fixtures and a Bedrock stub

The public project can be installed, built, tested, and run locally without private FedFathom source or paid cloud services:

https://github.com/alex-luy-assoc/agentic-market-memory

Most importantly, the project addresses a real workflow. It helps small contractors move from millions of historical federal records to a focused, reusable, evidence-backed market perspective.

What we learned

We learned that agentic memory is not simply chat history plus embeddings. Production-grade memory needs provenance, consent, trust, scope, lifecycle state, supersession, deletion, authorization, and a record of exactly how it affected an action.

We also learned that vector retrieval and transactional state belong together. Similarity can identify potentially relevant context, but transactions, tenant filters, immutable snapshots, and authorization determine whether that context is safe to use.

Deterministic software and generative models work best when their responsibilities are sharply separated. FedFathom selects evidence, calculates metrics, applies filters, and validates citations. Bedrock turns a bounded evidence package into readable narrative. This is more reproducible and defensible than asking a model to conduct the research itself.

Finally, resilience is part of agent behavior. If a worker can lose its lease, resume from a checkpoint, and still produce exactly one trusted version, recovery becomes an observable feature of the agent rather than an invisible infrastructure concern.

What's next for FedFathom: Agentic Market Memory

Our next step is to validate the Capability-to-Market Fit Brief with small federal contractors and measure whether it helps them make faster, better-supported market-focus decisions.

We plan to expand carefully into named research profiles, saved market monitors, change briefs, alerts, collaboration, and bounded exports.

A future customer-facing FedFathom MCP could let approved external agents use the same evidence and authorization contracts without receiving direct database access.

We will also continue improving vector-retrieval benchmarks, recovery testing, cost controls, observability, deletion workflows, and representative-user validation.

Any expansion will preserve the central trust boundary: memories remain user-controlled, evidence remains traceable, and models never silently turn untrusted content into durable research policy.

Built With

Share this project:

Updates