Hao-Med

Internet is a social determinant of health. A tiered healthcare meeting platform to provide service across all levels of internet connectivity.

Comment

Inspiration

You have a rash. It’s itchy, inflamed, and kind of puffy. ChatGPT tells you it’s life-threatening. Gemini tells you it’s fine. It’s extremely confusing, and in the midst of all of it, you don’t know if you’re fine or not. You call your doctor on the phone, but they tell you to make an appointment, because they cannot see you. But the problem is, you live 18 miles away. You get on Zoom. It crashes. You’ve lost the connection. And the earliest time you can book an appointment is in three months. Access to broadband Internet service (“broadband”) has recently been identified as a social determinant of health.

We can’t let them keep losing the connection. It's important for our culture and community, and especially detrimental to those who are rurally isolated.

Tying to the important of culture & community, we present Hao-Med, with "hao" being Khmer for "call", connecting to our Cambodian heritage (and paying an homage to our relatives who went through this telehealth struggle).

UX/UI Research

To lend to this, we wanted to ask users what do they look for to clarify telehealth appointments.

We surveyed 15 people from diverse backgrounds, and 66% identified 'lack of human connection' as their biggest technical fear. Our avatar system was designed specifically to solve that found insight.

Summarization: 46% of survey respondents indicated a need for a summary or detailed and simplified report at the end of their call to enhance clarity. 45.5% of survey respondents reported using landline calls for telehealth appointments, in which 76.9% reported that this rendered them unable to show physical symptoms, deterring them from needed telehealth appointments.

Survey respondents that once lived in rural areas wanted to find a way to send medical inquiries offline.

An app making telehealth more accessible would benefit a wide audience “interest in using telehealth to increase access to health and mental health care has grown” (Graves et. al). “Among responding districts, 64.2% (n=172) were rural and 35.8% (n=96) were urban. Rural districts reported significantly fewer students with access to an Internet-enabled device adequate for online learning (80.0% vs. 90.1%,p<0.01). Access to reliable broadband varied significantly across geography” (Graves et. al)

What it does

It is a hybrid care platform for video calls and logging health information, but not personally identifiable records. It combines synchronous (Live calling) and asynchronous (store it and forward it) technology, as well as AI summaries.

It allows a patient to record symptoms, take photos, and write notes locally on their phone, which the app then "bursts" to the doctor the second it finds a sliver of signal (to send medical inquires offline).

Instead of the call dropping, the app morphs the UI based on available bandwidth and connectivity speed. Tier 1: Full Synchronous (1 Mbps+): Standard HD Video. Tier 2: High-Compression (when at 250–500 kbps): Video resolution drops to 240p, and the frame rate is cut to 15 fps. The UI displays a "Data-Saver Active" badge to manage expectations. Tier 3: The "Avatar" Leap (when at 50–100 kbps): A video feed would be sending 15 frames per second, which uses a lot of data. The interface turns asymmetrical; the doctor’s video feed on the client’s side turns into an 2D vector avatar, which utilizes metadata and coordinates that are sent over as tiny lines of code over the network to animate the facial features. This is then fulfilled with Gemini Nano, locally on device. Tier 4: This is the highest level of data-preservation and technological innovation. The video feed is cut entirely. The app sends only a compressed audio stream and small metadata packets. If connection drops entirely, it records what you’re trying to say or put through the chat and bursts it over the second it regains signal. The "Found" Logic: The patient still sees the doctor "talking" via the avatar, maintaining the psychological human "Found" connection even when the actual video is "Lost."

How we built it

WebRTC: For the high-speed version (when the user is at the library or has 5G), the app uses WebRTC (Web Real-Time Communication). This is the same industry-standard technology used by Google Meet and Discord. We used a WebRTC-based architecture so the user doesn't have to download a heavy 100MB app; they can access it instantly via a lightweight mobile browser. Service Workers: These are scripts that run in the background. Even if the app is closed and the phone is in a pocket, the Service Worker is "finding" the signal. When it sees a bar of service, it "bursts" the doctor’s notes or the patient’s recorded symptoms. SQLite: This is a tiny, lightweight file on the phone that stores all the "Found" data (prescriptions, meeting history, and the map of hotspots) so it's always available instantly without a loading spinner. Gemini: is locally on device, and if on Android phones, is integrated into the chip. Provides summaries and real-time jargon translation during meetings, as well as avatar motion.

We designed it on Figma.

Challenges we ran into

Originally, the vector in the shared medical canvas feature was a 3D model; we realized this would add unnecessary weight to the application, which would be detrimental to the target audience.

It was going to be running entirely on browser, but we knew that the UI of browsers on phones are usually hard to work with. Also, statistically, people in rural areas are more likely to be entirely smartphone dependent for any digital or internet access.

Speech bubbles in the chat are blue. In an iOS 18 survey, a discrepancy for thinking of green text bubbles as lower was quite notable, and prevalent across most messaging platform users.

We were going to model it around used standard video degrading software, but we found this to be too inefficient and unoriginal for our purpose. This avatar system doesn’t exist on the current mainstream market!

We planned to pitch it to use MediaPipe face tracking for the avatar but switched to using Gemini because it’s more empathetic. Also, when the signal drops, using CV will freeze or lag, because it relies on real-time input. However, Gemini can fill in the signal gaps and simulate the movement.

Accomplishments that we're proud of

One of the main reasons telehealth calls stay long is the doctor explaining things. If the video drops, the app opens a Shared SVG Canvas, which we are proud of thinking of!

  • The doctor can circle an area on a medical diagram on their screen. (where we could draw this and create OG artwork)
  • Because the diagram is already cached (stored) on the patient's phone, the app only needs to send the XY coordinates of the doctor's drawing.
  • Technical Stat: Sending coordinates takes less than 1 KB of data, whereas showing a doctor's hand pointing on a video takes thousands of KBs.

Switching to Avatar Mode reduces data requirements by 90% compared to standard HD video. Cost Impact: A 10-minute call on Zoom costs a user ~150MB (Chan). A 10-minute "Avatar" call on Hao-Med costs under 10MB. No more wondering in confusion! Success Rate: In "Chaos" zones (3G/2G), Zoom has a 95% failure rate (Chan). Lumen's Tier 2/3 has a better chance of message delivery.

EMPHASIS ON THE ASYMMETRICAL INTERFACE: Even if it degrades to only avatar mode on the client's end, the doctor remains on the highest functionality (both HD video feeds) if their internet allows, tailoring each experience to the user.

What we learned

This was our first time designing a project and using Figma! We are proud to have learned of the User Research process and creating a project to solve a widespread problem that is neglected.

What's next for Hao-Med

We want to improve the system, and create a way for the doctor to immediately press SOS on the app and send 911 to your location, in case the patient collapses on screen.

We also want to add a scalable vector graphic to the medical dashboard that allows you to zoom in even farther to internal systems to improve visual understanding and interpretation of medical data.

For the avatar, we want to make it fully customizable to enhance the cultural and community value of the platform.

Built With

  • figma
Share this project:

Updates