Elderly Assistant (EA)

Inspiration

While Malaysia is fast becoming an ageing nation and driving digital public services (like STR/BSH), many tech-illiterate seniors are left behind. They don't know how to find official websites and feel genuine anxiety facing complex web forms. Consequently, many vulnerable elders fail to access critical financial aid, worsening their financial distress in paying for daily necessities and medical healthcare. Our project aligns with Malaysia's digital inclusion goals. We replace scary interfaces with an ultra-simplified, voice-first mobile assistant, helping seniors seamlessly secure government welfare to ease their financial burdens. We ensure our rapidly growing senior population moves forward with the nation, not left behind.

What it does

Elderly Assistant (EA) is a mobile-first, voice-driven platform designed to break the digital barrier for Malaysia's rapidly growing senior population, particularly within the vulnerable B40 community. Instead of forcing tech-illiterate elders to navigate multi-layered, English-dense official websites, our application features a "Zero-UI" philosophy—dominated by a single, accessible microphone button. Seniors can apply for critical government financial aids (such as STR and BSH) simply by speaking naturally in their daily local dialects and snapping photos of their identity documents. By transforming terrifying online forms into a friendly, localized conversation, we help vulnerable seniors seamlessly secure the government welfare they deserve, directly easing their financial distress regarding daily necessities and medical healthcare.

How we built it

We architected a streamlined, high-empathy system focused on extreme accessibility: ->The Interface: Built with a minimal frontend layout using modern web frameworks, stripped of complex navigation trees to prevent user anxiety. ->Multi-Modal AI Pipeline: Integrated advanced Large Language Models (gemini-3-flash-preview) to accurately capture entity data from conversational spoken audio and document scans. ->Context Control & Pacing: Engineered the backend to process information in discrete steps, prompting user validation and confirmation every 3 turns to minimize cognitive load. ->Human-in-the-Loop (HITL) Integration: Structured the extracted user data directly into standardized government PDF payloads, queuing them to a secure admin portal for public sector verification officers to review and approve.

Challenges we ran into

Building an infallible system for an audience with zero tech-literacy presented massive hurdles: ->The AI Token & Latency Wall: Continuous testing heavily exhausted our API quotas, leading to sudden response time-outs. We had to aggressively optimize our prompt structures and design a strict fallback system to prevent the application from freezing on the elderly users. ->The Precision Guardrail: In public welfare, data hallucinations can disqualify a vulnerable applicant. Fine-tuning prompt behaviors to aggressively ignore background noises/chatter while precisely extracting financial brackets required relentless trial-and-error. ->Integration and Compliance Barriers: Legacy government portals are historically anti-scraping and pose data privacy risks. We overcame this deadlock by pivoting our business logic toward a public-sector API partnership, utilizing our Human-in-the-Loop (HITL) architecture to guarantee absolute cybersecurity compliance.

Accomplishments that we're proud of

We successfully built a highly empathetic, working prototype that addresses a crucial national milestone—aligning perfectly with Malaysia's Digital Transformation and Inclusion Goals. We are proud of engineering a system where technical complexity is completely hidden under the hood, leaving the end-user with nothing but a simple, empowering, and dignified experience.

What we learned

This hackathon taught us that true innovation is often about subtraction rather than addition. Building for seniors taught us profound empathy; standard elements like endless chat logs that younger developers take for granted can cause severe anxiety for an elder. We also learned how to rapidly pivot our business model under intense time constraints when faced with integration and backend bottlenecks.

What's next for Elderly Assistant

Moving forward, we aim to expand our multi-turn conversational core to natively support localized broken-language patterns and deeper regional dialects (such as Manglish, local Malay slangs, and Chinese dialects). Furthermore, we plan to formally pitch this Human-in-the-Loop integration model to agencies like MDEC and the Ministry of Digital to establish a secure, official pathway, ensuring that as Malaysia marches into a digitized future, our seniors are moving forward with the nation—not left behind.