Dr. T Vision Alert — OpenCV 5 × AWS

Inspiration A health reading is only useful when it can be read correctly, understood in context, and acted on safely. Digital blood-pressure monitors and pulse oximeters display important measurements, but users may struggle to read small digits, interpret unfamiliar values, or transfer readings into a digital health record. Glare, blur, angled photographs, and partially obscured screens make the process even harder. We created Dr. T Vision Alert to bridge the gap between physical health-monitoring devices and digital health support. Our goal is to make visual health readings easier to capture and review while keeping people—not AI—responsible for confirming their measurements. What it does Dr. T Vision Alert adds a computer-vision workflow to Dr. T HealthAlert Mini, initially focusing on digital blood-pressure monitors and pulse oximeters. Users can capture or upload a device-display image. The vision pipeline assesses image quality, identifies the display region, corrects perspective where supported, and prepares the image for reading extraction. The system extracts candidate measurements, including systolic and diastolic blood pressure, oxygen saturation (SpO₂), and pulse rate when displayed and reliably readable. Before any extracted measurement enters the health workflow, the user must review, correct if necessary, and explicitly confirm it. Unreadable or ambiguous images lead to a retake or manual-entry option rather than an invented measurement. Confirmed readings can then feed into Dr. T's existing health intake and safety workflow. Deterministic safety rules take precedence over AI-generated explanations, and supported handoff functionality helps organize information for a healthcare professional. The system is designed as a health-support tool, not a diagnostic device. How we built it We integrated Vision Alert into the existing Dr. T application rather than creating a disconnected demonstration. Computer vision: The image-processing pipeline includes blur assessment using Laplacian variance, luminance and glare checks, contrast assessment, display-region localization, and perspective correction. These steps help determine whether an image is suitable for further processing. Reading extraction: OCR and structured parsing identify candidate values for supported monitor types. Plausibility checks help flag suspicious or incomplete values, but a plausible value is not automatically considered correct. Human confirmation: A review interface presents candidate readings alongside the source image, with editable fields and explicit confirmation. Data provenance distinguishes machine-extracted, user-corrected, and manually entered values. Safety-first integration: Confirmed measurements can flow into the existing HealthAlert Mini workflow. Deterministic safety rules remain independent of generative AI, and the integration preserves the intended SBAR clinician handoff and FHIR R4 risk-assessment workflow where supported. AI assistance: Gemini is used for secondary educational context and clinician-briefing assistance, with timeout and fallback handling. It does not replace measurement confirmation or override deterministic emergency rules. Cloud architecture: We prepared AWS Lambda infrastructure targeting ARM64, together with deployment definitions for CloudFormation and Terraform. The deployment package is intended to support a cloud-based processing path. AWS provisioning and live AWS execution must be distinguished from locally verified application behavior. The application uses a TypeScript-based web stack, with Python packaging for the AWS vision handler. Challenges we ran into Making images readable is not the same as reading them correctly. A photograph can be sharp but still contain glare, reflections, perspective distortion, or a partially obscured digit. We addressed this by treating image quality as an explicit stage instead of sending every image directly to extraction. Avoiding plausible but incorrect measurements. A misread blood-pressure digit can change the interpretation of a reading. We therefore designed the workflow around candidate extraction, plausibility checks, uncertainty handling, and mandatory human confirmation. Separating AI assistance from safety decisions. Generative models can be useful for explanations, but they must not delay or suppress emergency guidance. The design keeps deterministic safety rules separate from the model-assisted explanation layer. Supporting multiple execution environments. A browser-based pipeline and a Python-based AWS Lambda implementation have different dependencies and runtime constraints. Maintaining consistency across them requires independent version verification and testing. Benchmarking honestly. ARM64 and vectorized image-processing optimizations are architectural goals, not proof of measured speed improvements. We document externally reported performance separately from benchmarks measured in our own environment. Accomplishments that we're proud of We built Vision Alert as an integrated health-support feature with a complete intended journey from image capture to review and confirmed measurement import. Our implementation and verification work includes:

• Image-quality checks and display-localization logic.
• Candidate extraction and physiological plausibility checks.
• A mandatory confirmation gate with correction and provenance handling.
• Deterministic emergency escalation that does not depend on generative AI.
• Integration with the existing HealthAlert Mini workflow.
• AWS infrastructure definitions and ARM64-oriented packaging.
• Automated Vision Alert assertions and broader platform test coverage.
• Documentation covering architecture, deployment, privacy, testing, limitations, and judge demonstrations. The latest local audit reported 51 of 51 Vision Alert assertions passing, along with successful lint, build, and platform-test commands. These are software verification results, not proof of clinical accuracy or universal device compatibility. We are particularly proud of the design principle behind the feature: automation can assist with reading a measurement, but it should not silently turn an uncertain visual interpretation into a confirmed health record. What we learned Computer vision in healthcare requires more than a successful image-processing algorithm. Image quality, uncertainty, user interaction, data provenance, and safe failure behavior are equally important. We learned to distinguish image enhancement from OCR, OCR from measurement validation, and measurement validation from clinical interpretation. These are separate stages, each with different failure modes. We also learned that a health application must remain useful when a camera, OCR engine, network connection, or language model fails. Safe manual entry and deterministic safety handling are essential fallback paths. Finally, transparent engineering claims matter. A passing unit test does not establish real-world accuracy, an ARM64 configuration does not prove acceleration, and an infrastructure template does not prove a live deployment. Each claim needs evidence appropriate to what it asserts. What's next for Dr. T Vision Alert — OpenCV 5 × AWS Our next priorities are to verify and expand the supported-device test set, evaluate reading extraction against labelled real-world images, measure performance on the actual target hardware, and complete authorized AWS deployment and live end-to-end verification. We plan to improve handling of varied monitor layouts, reflections, low-light photographs, and ambiguous digits. We also aim to strengthen accessibility, multilingual guidance, privacy controls, and reproducible evaluation so that users can understand both the system's results and its limitations. Longer term, Vision Alert could become part of a broader personal health workflow that helps people organize measurements and share clearer summaries with healthcare professionals. Our guiding principle remains simple: make health readings easier to capture, safer to confirm, and more useful to discuss—without pretending that AI replaces clinical judgment.

Built With

• amazon
• arm64
• computer
• computing
• digital
• edge
• fhir
• healthcare
• imaging
• lambda
• learning
• machine
• medical
• ocr
• opencv
• optical
• processing
• python
• r4
• react
• recognition
• serverless
• typescript
• vision