SuianCare

• 

  Proactive care, leaving no loved one unseen.

Inspiration

Many older adults want to live independently, but everyday risks are easy to miss. A traditional reminder can say, “Take your medicine at 8 PM,” but it cannot tell whether the medicine was actually taken. The same problem appears when someone leaves the stove on, when a child has not arrived home safely, or when a pet shows unusual behavior.

We built SuianCare to move AI beyond passive question answering. Instead of waiting for someone to ask for help, it continuously understands daily activities, detects missed tasks or potential risks, and provides timely, evidence-based reminders.

What it does

SuianCare is a privacy-first multimodal care agent for older adults, children, pets, and their families.

Users can describe a care need in natural language or upload a PDF or image. SuianCare converts that information into an observable task containing goals, events, timing constraints, and alert conditions.

It then uses a phone, fixed camera, or wearable camera to understand what is happening in the real world. For example, it can determine whether an older adult has taken their medication, whether the stove was turned off, or whether a daily routine was completed.

When SuianCare detects an omission or risk, it proactively sends a voice, vibration, or WeChat notification. The alert can include an evidence frame so that the user or caregiver understands why it was triggered. Users can also provide text, image, or voice feedback to refine the task over time.

How we built it

SuianCare combines multimodal AI, an agent runtime, multiple client applications, and edge hardware.

The backend is built with Python, FastAPI, Pydantic, and an OpenAI-compatible multimodal model API. Separate agent capabilities handle task generation, visual reasoning, short-term memory, feedback-based task refinement, and notifications.

The main client is a native WeChat Mini Program. We also built an iOS application with SwiftUI and AVFoundation, as well as a wearable prototype using the XIAO ESP32S3 Sense camera board.

Instead of continuously streaming video, the client uploads individual frames only when needed. After analyzing each frame, the server recommends when the next frame should be captured. This adaptive sampling approach reduces bandwidth, energy usage, unnecessary image collection, and inference cost.

For production deployment, the system supports PostgreSQL, Redis, MinIO or S3-compatible storage, Docker Compose, HTTPS, request auditing, rate limiting, and encrypted evidence storage. We also created a simulation and evaluation environment for testing household scenarios before real-world deployment.

Challenges we ran into

The hardest challenge was determining whether an event actually happened from a sequence of incomplete visual observations. A single frame may be ambiguous, so the agent must combine the current image with the task definition, recent events, and selected historical keyframes.

Another challenge was balancing response speed, model cost, device power consumption, and privacy. Continuous video streaming would provide more information, but it would also collect unnecessary data and consume significant bandwidth. We addressed this with adaptive frame sampling and bounded visual memory.

False alarms were also an important concern. We introduced confidence thresholds, alert cooldowns, ordered frame processing, stale-result rejection, and user feedback so that the system can become more accurate without repeatedly disturbing the user.

Finally, supporting multiple clients and real hardware required us to design a consistent API while handling unreliable networks, concurrent sessions, authentication, evidence ownership, and secure model access.

Accomplishments that we're proud of

We built a complete working loop rather than a standalone AI demonstration:

1. A user describes a real-life care requirement.
2. The agent converts it into a structured, observable task.
3. A phone or wearable device captures the scene adaptively.
4. The multimodal agent evaluates task progress and possible risks.
5. The system proactively alerts the user or family.
6. User feedback can refine the task without recreating it.

We are also proud that SuianCare works across a WeChat Mini Program, an iOS client, fixed cameras, and wearable hardware. The project includes a deployable backend, family collaboration, evidence-based alerts, privacy controls, and a simulation framework for measuring false positives and missed alerts.

What we learned

We learned that proactive AI requires more than a powerful vision model. It needs reliable temporal reasoning, carefully bounded memory, clear alert policies, privacy-aware data handling, and an interface that people can trust.

We also learned that an effective care system should not try to replace family members or professional caregivers. Its role is to notice important situations earlier, reduce repetitive monitoring work, and help people respond with better context.

Most importantly, accessibility must influence the entire workflow. Voice input, simple interactions, automatic task structuring, and clear reminders are essential when designing for older adults and other users who may find complex interfaces difficult.

What's next for SuianCare

Next, we plan to validate SuianCare with more real household scenarios and measure false-alert and missed-alert rates systematically. We will improve on-device privacy processing, offline fallback behavior, and support for additional wearable sensors.

We also plan to expand family and community collaboration, introduce escalation paths for high-risk events, and work with care organizations on small-scale pilots. Longer term, SuianCare will learn from consented user feedback to provide increasingly personalized and reliable support while keeping users in control of their data.

Built With

• ai-agents
• esp32
• iot
• multimodal-ai
• redis
• vllm
• wechat-mini-program