권리시그널 (Rights Signal)

An evidence-first agent that helps social-service case managers review corrected benefit notices and reopen only cases that need human attention.

Comment

Inspiration

When a public-benefit notice is corrected, social-service case managers face a risky choice: review every completed case again or rely on a manual comparison that may miss someone. Generic AI summaries do not solve the accountability problem. Workers still need to know which source clause supports each suggestion, what information is missing, and who made the final decision.

Rights Signal was built for that gap: reducing repeated review work without turning a language model into an eligibility decision-maker.

What it does

권리시그널 (Rights Signal) turns a corrected housing-support notice into a controlled, evidence-first review workflow.

A read-only Strands agent compares versioned source sections and proposes evidence-linked rule and correction candidates. A case manager verifies those candidates against the original notice. Only then does deterministic code calculate MET, NOT_MET, or UNKNOWN, identify affected cases, and block completion when evidence is incomplete.

In the demo, a residency-clause correction reopens exactly 2 of 12 synthetic cases while preserving the other 10. One reopened case is missing a residency start date, so it remains UNKNOWN. After the worker checks the source and records the confirmed date, that condition changes to MET; the worker—not the model—completes the review. Every step is retained in an append-only audit trail.

Why it matters

Social-service organizations need automation that reduces repetitive work without hiding uncertainty or transferring public-impact decisions to a model:

  • AI compresses long documents into reviewable, source-linked candidates.
  • Deterministic code performs eligibility prechecks and correction-impact calculations.
  • A human confirms rules, resolves missing facts, and makes the final review decision.

How we built it

The React and Vite workbench is hosted on AWS Amplify. Amazon API Gateway routes requests to a Python 3.12 AWS Lambda application. Lambda coordinates deterministic domain services, DynamoDB transactions and audit records, S3 notice fixtures, and a read-only agent built with the Strands Agents SDK. The agent uses Amazon Bedrock Nova Micro for structured language analysis. CloudWatch records request and tool-call evidence without storing sensitive fact values.

The agent has three read-only tools:

  1. read_notice_sections reads anchored sections from versioned notices in S3.
  2. read_confirmed_rules reads the current human-confirmation state.
  3. compute_section_diff returns a deterministic text diff between notice versions.

The agent cannot write case data, confirm a rule, decide MET / NOT_MET / UNKNOWN, select affected cases, or complete a review. Invalid structure, missing anchors, or a model failure produces an explicit failure or a clearly labeled prepared fallback; it never silently changes case state.

Key features

  • Source-linked agent analysis with stable notice anchors
  • Human confirmation before any proposed rule becomes active
  • Three-valued evaluation that keeps missing facts as UNKNOWN
  • Targeted correction review that reopens only affected cases
  • Immutable evaluations, signoffs, idempotent writes, and audit events
  • Failure-safe model path with an honestly labeled prepared fallback
  • Shared operational workbench for evidence, conditions, and history

Challenges we ran into

The hardest design problem was not extracting rules; it was deciding what the agent must never decide. We separated probabilistic language analysis from deterministic evaluation and human approval, then tested that boundary through API contracts and end-to-end correction scenarios.

A second challenge was making the shared demo reliable. We added idempotent writes, deterministic synthetic fixtures, reset and verification scripts, and explicit fallback labeling so model or infrastructure failures never masquerade as a successful live analysis.

Accomplishments that we're proud of

  • A live Strands and Nova analysis path with source-linked structured output
  • Targeted impact calculation that reopens exactly 2 of 12 cases
  • Honest UNKNOWN handling that blocks completion until a worker verifies the source fact
  • An append-only audit path from rule confirmation through final human signoff
  • A deployed public demo backed by repeatable local and AWS verification

What we learned

Agentic systems become more trustworthy when they are given narrow tools, explicit non-authority, validated output contracts, and a product interface that shows uncertainty instead of smoothing it away. The model is most useful here as an evidence-navigation assistant, not as the final decision engine.

What's next

Next steps include supporting more notice formats and benefit programs, adding organization-level access control, measuring reviewer time saved, and evaluating Amazon Bedrock AgentCore for stronger runtime isolation and observability.

Built with Codex

Codex was the primary coding collaborator from scope through deployment. It helped develop the PRD and technical specification; implement the React/TypeScript frontend, Python domain services, API contracts, Strands integration, and AWS SAM infrastructure; write regression tests; diagnose deployment issues; and organize submission evidence.

Human review remained part of every high-impact choice, including product scope, the AI/human boundary, AWS changes, interface language, and release readiness.

Testing the live demo

The public demo uses synthetic data only.

  1. Start change analysis and inspect the live Strands/Nova result beside the original notice evidence.
  2. Review and confirm the proposed v2 rules.
  3. Open the impact preview and confirm that C-003 and C-009 reopen while 10 unaffected cases stay completed.
  4. Open C-003; its residency condition is UNKNOWN because the start date is missing.
  5. Enter the source-confirmed date 2024-01-10; the condition changes to MET.
  6. Complete the v2 precheck and inspect the immutable audit history.

Limitations

This hackathon demo covers one synthetic housing-support program and does not connect to real government records. It is a pre-review workbench, not an official eligibility determination, legal opinion, or automated application service. The public demo must never receive real personal data.

Built With

Share this project:

Updates