• 

• 

BoneTwin — Digital Twine Bone Health Records

Inspiration

Bone health is followed over many years, but reports often end up scattered across different offices, systems, and devices. One report may measure a different part of the body, and a doctor may decide that a particular result should not be used in future comparisons. By the time the next scan arrives, that important decision can be difficult to find.

We created BoneTwin to give the record a reliable memory. It keeps reports together and remembers which details were trusted, corrected, or still need attention. When a new report is added, BoneTwin uses the relevant history, leaves out information that was previously marked unsuitable, and clearly explains why. If a reviewer approves or corrects something, BoneTwin remembers that choice the next time the record is opened.

Our goal is not to replace a doctor or make a diagnosis. BoneTwin helps people organize their history, understand what changed, and prepare better-informed questions for human review.

What it does

BoneTwin turns a set of fabricated or fully de-identified DXA reports into a source-backed, longitudinal record whose corrections and review decisions persist across sessions.

The end-to-end experience works like this:

1. A user enters the demo and explicitly loads a pseudonymous subject record. The dashboard does not make unnecessary Lambda calls when it first opens.
2. The user selects a PDF. BoneTwin hashes the exact browser bytes and creates an authenticated, subject-scoped, idempotent upload intent.
3. The browser uploads directly to a private Amazon S3 object through a short-lived SigV4 URL. The request includes checksum and SSE-KMS requirements.
4. AWS Lambda verifies and parses the report. The application preserves the original source text, normalizes supported measurements, records parser provenance, and creates safe memory summaries.
5. Amazon Titan Text Embeddings v2 generates normalized 1,024-dimensional embeddings.
6. A CockroachDB transaction stores the report, measurements, source evidence, vector memory, workflow state, and audit event. The temporary raw S3 object is deleted after the durable commit, with lifecycle expiration as a backstop.
7. When the user runs Trusted Comparison, LangChain calls the CockroachDB Cloud Managed MCP Server. BoneTwin allowlists only select_query and uses the returned IDs to gate which memories may enter agent context.
8. CockroachDB Distributed Vector Indexing supplies subject-prefixed semantic candidates. The Memory Trust Engine then applies deterministic scope, verification, validity, and supersession filters.
9. Amazon Bedrock Nova Lite receives only the authorized evidence and produces a strict structured decision. Any cited memory outside the authorized set is rejected. The model cannot execute a database write or approve its own action.
10. The Overview displays the result beside the append-only Backend Processing Trace. The full Memory Impact Trace shows used and excluded memories, trust scores, exclusion reasons, and a counterfactual describing how the result would differ without the key memory.
11. If a bounded review task is proposed, a human can approve, correct, or reject it. One transactional write records the task decision, audit event, and verified or rejected memory.
12. In a new browser session, BoneTwin retrieves the earlier decision from CockroachDB and changes the next action without being reminded.

The result is not merely a searchable report archive. CockroachDB stores the operational facts, vectors, task state, corrections, run traces, and audits that determine what the agent is allowed to do next.

Product flow

flowchart LR
    A[User uploads a fabricated DXA PDF] --> B[Private S3 upload]
    B --> C[Lambda validates and parses]
    C --> D[Titan v2 creates embedding]
    D --> E[(CockroachDB stores evidence and memory)]
    E --> F[LangChain calls Managed MCP select_query]
    F --> G[Scope and trust filters]
    G --> H[Bedrock returns structured proposal]
    H --> I[Application validates evidence and policy]
    I --> J[Human review]
    J --> K[(Verified memory and audit)]
    K --> L[New session retrieves decision]
    L --> M[Later action changes]

What makes the memory different

Ordinary document RAG	BoneTwin
Retrieves text that looks similar	Retrieves subject-scoped candidates and then applies trust policy
Often treats retrieved chunks equally	Tracks provenance, verification, validity, and supersession
Produces free-form output	Requires a strict, evidence-authorized decision schema
Lets model output drive the experience	Lets the model propose while application code authorizes and commits
Usually hides retrieval decisions	Shows used and excluded memories with reasons
May forget a correction in a later session	Stores the human decision as durable CockroachDB memory
Explains an answer	Explains which memory changed the action and provides a counterfactual

How we built it

BoneTwin is a TypeScript/Python monorepo. The responsive client is built with Next.js and React; the API is FastAPI on Python 3.12. The hosted hackathon slice uses AWS Amplify Hosting for the web application and an allowlisted same-origin proxy, while FastAPI runs through a Lambda Function URL.

CockroachDB Cloud is the system of record rather than a passive document store. Relational tables hold reports, measurements, immutable correction chains, tasks, reviews, agent runs, retrieval dispositions, and audit events. Safe memory summaries are stored beside VECTOR(1024) embeddings. The vector index is prefixed by tenant and subject constraints so semantic retrieval begins inside the authorization boundary instead of filtering another user's candidates afterward.

Transactional SQL handles all state changes because report ingestion and review resolution span multiple tables and must be idempotent and audited. Managed MCP is intentionally read-only in the agent path. LangChain connects to the CockroachDB Cloud MCP endpoint, but BoneTwin exposes only select_query; tools advertised by the server are never automatically passed to the model. MCP returns authorized memory IDs, and application code performs the final retrieval and trust policy.

Amazon Bedrock serves two bounded roles:

• Titan Text Embeddings v2 creates normalized 1,024-dimensional vectors for semantic memory retrieval.
• Amazon Nova Lite creates a schema-constrained comparison proposal over evidence already authorized by the application.

Every model response used by application logic is validated against a strict schema. Evidence IDs must belong to the retrieved subject scope, actions must come from a small allowlist, and medical output policy rejects diagnosis, treatment, and fracture-risk recommendations. Free-form model text never executes a tool. If the live model produces a response that fails strict evidence validation, BoneTwin fails closed or uses a clearly labeled bounded fallback over the already authorized evidence.

Document bytes use a separate short-lived path. The browser uploads directly to private S3 with a signed URL, checksum, and KMS encryption. Lambda reads only the scoped object, verifies integrity, and deletes it after the CockroachDB transaction succeeds. Database credentials and MCP secrets remain in Secrets Manager and never reach the browser. CloudWatch and X-Ray provide runtime logs and request tracing.

The custom BMD parsing concept and earlier Bone Health Tracker work predate this hackathon and are disclosed in NOTICE-PREEXISTING.md. The new hackathon work is the CockroachDB memory model, trust-aware hybrid retrieval, LangChain/MCP boundary, Bedrock decision contract, human-review transactions, Memory Impact Trace, evaluation suite, AWS deployment, and polished demo workflow.

Deployed architecture

The diagram above represents the services exercised by the hosted demo. Cognito, Textract, Step Functions, Comprehend Medical, and AgentCore exist only as future-ready contracts or infrastructure definitions and are not claimed as deployed.

AWS and CockroachDB deployment

flowchart LR
    Browser[Browser] --> Amplify[AWS Amplify Hosting]
    Amplify --> Proxy[Same-origin API proxy]
    Proxy --> Lambda[AWS Lambda + FastAPI]

    Browser -. signed PDF PUT .-> S3[Private Amazon S3]
    S3 --> KMS[AWS KMS encryption]
    Lambda --> S3
    Lambda --> Secrets[AWS Secrets Manager]
    Lambda --> Logs[CloudWatch + X-Ray]

    Lambda --> Titan[Bedrock Titan Embeddings v2]
    Lambda --> Nova[Bedrock Nova Lite]
    Lambda --> LangChain[LangChain MCP adapter]
    LangChain --> MCP[CockroachDB Managed MCP]
    MCP --> CRDB[(CockroachDB Cloud)]
    Lambda <--> CRDB

Trusted-memory decision flow

flowchart TD
    A[Authenticated subject scope] --> B[Structured timeline retrieval]
    A --> C[Distributed vector candidates]
    B --> D[Candidate memory set]
    C --> D
    D --> E{Memory Trust Engine}
    E -->|verified, valid, relevant| F[Authorized context]
    E -->|rejected, expired, superseded, out of scope| G[Excluded with reason]
    F --> H[Amazon Bedrock structured proposal]
    H --> I{Schema + evidence + safety validation}
    I -->|invalid| J[Fail closed or bounded safe fallback]
    I -->|valid| K[Proposed review task]
    K --> L{Human decision}
    L -->|approve or correct| M[(Verified CockroachDB memory)]
    L -->|reject| N[(Rejected memory + audit)]
    M --> O[Later session behavior changes]

Production-minded safeguards

• State-changing endpoints are authenticated, role-authorized, subject-scoped, idempotent, and audited.
• Original measurements and source evidence are append-only; corrections supersede rather than overwrite.
• MCP is read-only and allowlisted; the model receives no general-purpose SQL or write tool.
• Retry handling is bounded, and CockroachDB serialization conflicts are retried safely.
• Duplicate uploads and repeated task resolutions do not create duplicate durable state.
• Prompt injection and active markup are screened before downstream context.
• Cross-subject retrieval tests require zero protected-memory leakage.
• The public environment is restricted to fabricated demo records. Its fixed demo identity is not represented as production authentication.

Evaluation

We built 30 deterministic fabricated timelines spanning trust conflicts, superseded corrections, cross-subject traps, prompt injection, concurrent updates, and retry recovery. In the checked-in evaluation, hybrid trusted-memory retrieval achieved 100% key-memory recall and safe-action accuracy with zero cross-subject leakage. A latest-only baseline missed the key memory, while an unfiltered vector-only baseline retrieved unsafe trap memories. These results are a software and memory-quality evaluation, not a clinical effectiveness claim.

Challenges we ran into

• Making memory causal instead of decorative. We had to prove that a prior verified decision changed a later action, not simply display database rows beside an LLM response.
• Keeping model output inside the evidence boundary. Bedrock responses are rejected if they cite memory outside the authorized CockroachDB set, which required strict schemas and a bounded fallback path.
• Balancing a low-cost serverless deployment with responsiveness. Concurrent startup calls initially hit Lambda concurrency limits, so the dashboard now loads only on explicit user action and combines related reads.
• Preserving source truth while supporting corrections. Corrections had to be append-only and traceable without silently modifying extracted measurements.

Accomplishments that we're proud of

• A human-approved correction changes behavior in a completely new session.
• CockroachDB stores relational state, distributed vectors, task state, run traces, and audits in one consistent memory system.
• Managed MCP is meaningfully integrated through LangChain while remaining read-only and narrowly allowlisted.
• The UI exposes the real backend processing path and makes used, excluded, and counterfactual memory visible.
• The hosted vertical slice runs on AWS and passes automated safety, isolation, idempotency, resilience, formatting, typing, and production-build gates.

What we learned

• Agent memory becomes useful when it includes trust and lifecycle—not merely similarity.
• A model should propose an action, while deterministic application code authorizes and commits it.
• Counterfactuals make persistent memory much easier to evaluate and explain.
• Relational state and vector memory are safer when they share one transactionally consistent system of record.

What's next for TwinBone

• Replace the fixed public-demo identity with production identity and consent management.
• Add broader report-format support and reviewer-confirmed extraction workflows.
• Evaluate usability with patients, clinicians, and care coordinators without making clinical effectiveness claims.
• Explore multi-region resilience, richer observability, and larger subject-scoped retrieval benchmarks.
• Keep the core boundary unchanged: source-backed organization, transparent memory, and human control—never autonomous diagnosis or treatment.

Built With

• amazon-web-services
• amplify
• bedrock
• cockroachdb
• ecs
• s3