SAKSI

Inspiration

It started with a short comedy sketch: "POV: a blind person buying something." The joke was the cashier. He could say any price, hand back any change, and the buyer had to take his word for it. I laughed, and then I kept thinking about it.

In a shop there is at least a person standing there. Online there is no one. A blind user who shops or pays alone hears whatever the screen reader reads out, and has to trust that it read the right number, the right name and the right button. If something is off, nobody is there to say "wait, that's not right."

I started with online shopping. When we looked at the research, we moved to bills first, for four reasons:

• Bills have a number you can predict. Electricity, health insurance (BPJS) and internet come every month at roughly the same amount. That lets the user say what they expect before hearing the bill, and lets us compare it with their history. A shopping cart has no "expected amount."
• The risky step is checking, not choosing. A 2026 study of 103 people with long-term vision loss found they rely on others for money tasks, and worry about privacy and safety. In Indonesia, before the KCI commuter card app could read the balance aloud, blind riders needed someone next to them to read it. Paying a bill is almost pure checking: who, how much, from which account.
• Shopping adds a second problem. In a CHI 2026 study of a voice shopping agent (n=12), blind participants kept asking for double-checks and were afraid product details were misread. That study never tested the payment step. That is the step we took.
• The payment rails exist. Agent payment systems today (Alexa+ Payments, AP2) handle cards only. Most Indonesians pay bills with a virtual account (VA) or an e-wallet, and the Midtrans sandbox lets us run real VA payments without real money.

Shopping is still the goal. Bills are the part we could make safe first.

What it does

You set a limit once by voice: "max 2 million rupiah, all bills, until October 31, guardian above 1 million." Then:

1. Before paying. You say "pay my PLN bill." Saksi does not read the amount. It asks: "How much do you expect it to be?" You answer first. We call this a blind-check. It stops the habit of saying "yes" to whatever was just read out.
2. While paying. A gate on the server checks the limit, the biller, your expected amount, and the bill's history. If anything fails, the payment stops. If the bill is above your guardian limit, someone you trust approves it from their own phone without ever holding your PIN.
3. After paying. Saksi reads back what was paid. You have 5 minutes to cancel, and you can ask later: "What did I pay last week?"

If someone calls and says "pay 5 million now or your power gets cut," Saksi notices the pressure, slows down, and warns you before anything moves.

How we built it

• MCP server (Python). All payment logic lives here, not in the AI. It has two separate channels. The agent channel can only prepare a payment. The client channel is the only one that can approve, and it only takes your own confirmation from the app. So even if the model is confused, or a bill contains a sneaky instruction, the AI cannot pay on its own.
• Anomaly check. A simple rule plus Amazon Nova on Bedrock. With at least 3 months of history $h_1, \dots, h_n$, a bill of amount $a$ is flagged when

$$a > 1.5 \cdot \bar{h}, \qquad \bar{h} = \frac{1}{n}\sum_{i=1}^{n} h_i$$

Nova also reads the history and can flag a bill the rule misses. With less history, only Nova judges, and the receipt says "limited history." If the check itself fails, the payment is blocked. We would rather stop a good payment than let a bad one through.

• Agent. Strands Agents with Amazon Nova Pro, running on Bedrock AgentCore Runtime. Before Nova, a small router (Jev, on TypeSafe) sorts each sentence: off-topic gets a fixed reply, simple questions like "what bills do I have?" are answered straight from the server without calling Nova, and pressure from a third party switches on caution mode.
• Voice web client (Svelte + TypeScript). Hands-free: it listens until you pause, speaks back with Amazon Polly, and hears you through Amazon Transcribe. A trace panel shows every MCP call for the judges.
• AWS. Everything is deployed with AWS CDK: Cognito for login, DynamoDB for per-user state, Lambda, S3 + CloudFront for the site, Secrets Manager for keys, and a budget alert.

Challenges we ran into

• Alexa+ was not open to us. I assumed we could build directly on the Alexa+ MCP Toolkit. From Indonesia, the developer account failed identity verification, deploying an add-on gave AccessDenied, and Amazon Support finally told us the Toolkit is Private Preview for selected partners only. So we built our own MCP server (spec 2025-11-25) and a web client that plays the role of Alexa+. It is a simulation, and we say so in the demo.
• The sandbox payment never finished. A sandbox VA stays "pending" forever because nobody can really transfer money to it. We automated the Midtrans payment simulator so Midtrans itself records the settlement.
• GoPay was blocked. GoPay account linking returned "Merchant not authorised" for our sandbox account. The code path is there, but the demo uses VA (BCA, BNI, BRI) only.
• The model misheard and wandered off. In early tests Nova answered questions about Amazon and JavaScript, used markdown that sounds terrible when read aloud, and heard "BPJS" as "baby JS." The Jev router and stricter prompts fixed most of it.
• "Did it go through?" If the payment gateway times out, we don't know whether the money moved. Saksi keeps the same order ID and retries with it, so there is at most one charge, and it never tells the user "failed" unless the gateway proves it.

What we learned

• Moving a step to "a secure page" can hide it. The usual safety advice is to send sensitive steps to a trusted web page or app screen. For someone who can't see the screen, that moves the check to the one place they can't read. We kept every check in the voice conversation, and the account is linked only once, during setup.
• "Yes" is too easy. Asking "Pay 350 thousand to PLN?" mostly gets a yes. Asking "how much do you expect?" makes the user actually check.
• Don't let the AI approve anything. The model is good at understanding what you mean. It should never be the one saying yes to money. Splitting prepare and approve into two channels made that a rule of the system instead of a line in a prompt.

What's next

• Bahasa Indonesia voice.
• Real billers.
• Online shopping, the idea this started from.

Built With

• amazon-bedrock
• amazon-cloudfront
• amazon-cognito
• amazon-dynamodb
• amazon-nova
• amazon-polly
• amazon-transcribe
• amazon-web-services
• aws-cdk
• aws-lambda
• aws-secrets-manager
• bedrock-agentcore
• midtrans
• model-context-protocol
• pydantic
• pytest
• python
• strands-agents
• svelte
• typesafe
• typescript
• vite