Aperta

Explain a Note - Main Page
Example of Broken Wrist
Read Aloud Screen with Broken Wrist Example
What to do Next Screen
Additional Information for the Patient
Anatomy Library
Dark Mode Demo
Different Language Usage Example
Printout

Inspiration

Patients generally forget the majority of what they are told during a consultation and end up misremembering a large part of the remainder. The solution is to use a platform which is able to explain and store the information obtained from the doctor's visit. This process only takes a few minutes, not fifteen minutes. As a result, a visit that has been forgotten can soon be replaced by a simple instruction sheet that proves to be very helpful to the patient.

We consider the most valuable screen in medicine to be the one that the doctor directs towards the patient, since it eliminates confusion caused by misremembering.

What it does

The clinician sticks on the note which they have already prepared—this includes a discharge summary, an after-visit summary, a clinic note, and an EHR export complete with ICD-10 codes. Aperta puts together what the patient requires and displays each sentence to the clinician before the screen changes.

Where on me: A hand-drawn figure showing a red dot on the relevant area, followed by a magnified drawing of the relevant portion with the structure labelled as this bone and this socket. All of the drawings are drawn by hand, there are eighteen views, and each dot indicates a real structure.
What happened: A person is able to repeat back one sentence from a set of 205 conditions grouped into 21 categories.
For each medicine: there are clear instructions regarding both when and how frequently the patient should take it.
Before you make a prescription: On the same screen, the drug in question is compared with all the others regarding duplicates, labelled interactions, the boxed warning, use in specific populations, dosing, contraindications, and whether or not a generic is available. This information has been taken from the FDA label, together with a link back to it. – What to do next and how to carry it out: Nine step-by-step instructions (such as rinsing a tooth socket with a syringe, using an inhaler with a spacer, using crutches, applying eye drops, using a sling, applying a dressing, using an injection pen, using ice and elevation, and looking after a splint), which are automatically included when the note requests them.
For Spanish, Vietnamese, Chinese, and Arabic and in relation to the read aloud function: Each screen can make use of ElevenLabs to be read out aloud, together with other language options. Moreover, there is a print-out feature so that the HCP can share it with the patient.

How we built it

The site has been exported as a fully static website using Next.js 16 together with the App Router, TypeScript and Tailwind v4. The drug identity and class are obtained from RxNorm and RxClass, the labels and packaging from openFDA, and the translations are provided by MyMemory since it requires no key. For the read-aloud function, ElevenLabs clips were included at build time for the example notes, while the browser's own voice is used when a clinician pastes in text. Each drawing was created by hand in Procreate and then exported as black ink on a transparency, so that the file appears as black ink on paper and as beige ink on the dark board.

The website that has been published does not have a backend and therefore does not need one. The parser, the card builder, and the prescriber checker are simply ordinary TypeScript programmes running within the page, which means that any note copied into the demo link will work without having to enter a key, and the note itself never gets sent to any of our servers.

Four rules run through the code:

Parsing is carried out in a deterministic manner. This involves eighty section-header aliases, the ICD-10 prefixes for each condition, dose patterns, the pipe-delimited EHR fields, and a review of the notes which were written without any headers. A model only fills in missing entries when such a feature has been configured, and the demonstration version runs without any such configuration. – The library has the final say as to what a diagnosis is. For each of the 205 conditions there are included the terms that are actually used in clinical practice ("NSTEMI", "Colles", "HNP", "s/p extraction") together with their ICD-10 prefixes. The code takes precedence over any phrase, the longest phrase is preferred, and 'ST depression' on an ECG trace does not mean depression. The purpose of a medicine is determined by 95 curated class rules, the rules being based on ATC and EPC identifiers rather than on the names of the drugs, followed by a visit context layer in the cases where the class rule on its own would be misleading, and finally by the indication stated in the label. It is never necessary to ask your pharmacist since the doctor is present.
Translation merely translates. The drug names remain exactly as they are, since the patient needs to identify them with the bottle.

No clinical material is produced. It is the code and the library that determine which drawing, which sentence, and which warning a note becomes, and the clinician has to read all of it before the screen turns around.

Challenges

The note generated 48 slides, slide two being a symptom. The entry under Medications reading 'Headache (QOD)' was treated as a medication, and RxNorm's fuzzy matcher identified it as headache powder. Each of the steps in every how-to procedure had a slide allocated to it as well. A line will be considered a medication only if it matches as a drug order and the resulting resolution shares at least one word with the printed text; in all other cases the entry is returned to the clinician with the status Not used, and a walkthrough consists of a single screen with the steps numbered on it. That note takes up five screens.

There is no instance in which two notes are written in the same manner. Previous testers had pasted in things that the parser had never come across, such as "Medicine:" when what was expected was "Medications", signatures that began with a dash, prose without any headings at all, and a second note being pasted on top of the first so that the original prescription remained displayed on the screen. Each of these cases was included in a regression suite which tests every note format we have collected each time there is a change, meaning that correcting one note cannot do so without possibly breaking another.

Asking for 'real 3D anatomy' was wrong because photographic models of the organs come across as gruesome to a person who has just received bad news, and a third-party model can't have your fracture indicated on it. Therefore we gave up on 3D and produced all eighteen views ourselves. To get the drawings in place we needed a small pipeline of our own: we had to crop each drawing to the area with ink, recolour it to match the ink of the page, then print a labelled sheet showing the red spot for each condition on its corresponding drawing and check them by eye. The position of the red spot for a Colles' fracture is at the lower end of the radius since someone had looked at it.

The time was taken up by the design. It took three rounds to progress from something pleasant but ordinary to one that showed real skill: the interface was drawn in the same ink as the anatomy, the markings are made automatically, the screen actually turns, and as a note is read the spot searches the body and then lands with a bounce. The rest consisted of removing features such as cards, shadows, pills, and the native dropdowns, as well as the system's dark mode.

Accomplishments that we're proud of

The entire operation is carried out without any API keys, including for the live website; API keys provide improvements to voice quality, support for photo input, and a prior-auth letter, not a finding.

Each anatomical view is drawn by hand, with the presence of each condition being indicated on an actual structure.
A regression involving the parsing of notes for seven formats which is carried out with each change. Here’s a demonstration you can carry out: enter a symptom in the section on Medications and you’ll see it being sent back rather than heard aloud. – The turn. No one else's demo has the laptop turned around.

What we learned

The real issue wasn't about drawing accurate anatomical diagrams or naming the model; it was about refusing to make a guess in a field where a certain wrong statement could cause harm, and making that refusal clear so that both the judge and the clinician could have confidence in the rest. The second point is that craft consists of a series of removals.

What's next

Aperta can be obtained via an app. When Aperta is available as an app, patients are able to refer to their printed copies and related information on their phones in a more convenient manner than having to keep the printed copies. Moreover, the healthcare professional can alter any of the information if necessary or make any adjustments, with the changes being updated in real time.

This is an educational demonstration only and should not be regarded as medical advice; real patient data has not been used.