Aurasync

Onboarding Process
Homepage
Dashboard
chrome Extention
Diagram_Example
Tone+Speakers_Example

Inspiration

More than 5 to 7 percent of people globally, including nearly 1 in 10 children, experience Auditory Processing Disorder or similar auditory processing challenges. They can hear sounds clearly, but their brain struggles to organize, interpret, and respond to them, especially in noisy environments, fast conversations, or digital learning spaces.

What inspired AuraSync was realizing that most accessibility tools focus on accommodation rather than growth. They repeat or simplify information, but rarely help users improve how their brain processes sound over time.

During the Google Gemini 3 Hackathon, we asked a different question. What if AI could adapt with the user and then gradually step back once the brain starts succeeding on its own?

AuraSync was built on the belief that accessibility should lead to independence, not dependency.

What it does

AuraSync is an AI powered adaptive auditory processing platform designed for people with Auditory Processing Disorder.

It dynamically does the following. • Breaks down complex auditory information • Adjusts explanation depth, clarity, and pacing in real time • Tracks user behavior such as listening speed, pauses, and simplification requests

Over time, AuraSync helps users move from assisted listening to confident and independent understanding.

AuraSync can also understand tone such as sarcasm or happiness. It can detect how many speakers are present and in some cases infer roles like teacher and student based on context. It also detects background noise and intelligently reduces it to improve clarity.

How we built it

AuraSync was built as a full stack AI first system with accessibility and responsiveness at its core. We also built a Chrome extension, making AuraSync available across the web.

Frontend

The frontend was designed to be clean, minimal, and distraction free, with a strong focus on reducing cognitive load.

Every element was built to be accessible, intuitive, and usable by a wide range of users regardless of their processing abilities. The goal was simple. The interface should never compete with the user’s attention.

Backend

The backend handles session management, interaction logging, and adaptive decision making.

It captures behavioral signals such as listening speed, pauses, pacing changes, and simplification requests, and continuously feeds them into the AI system. This allows AuraSync to adapt in real time based on actual user behavior rather than fixed presets.

AI and Intelligence Layer

AuraSync is powered by Google Gemini 3, which acts as the core multimodal reasoning model.

Gemini 3 enables an agent based architecture that reasons over user onboarding data and real time interaction signals to adapt explanation depth, pacing, and modality. The system dynamically chooses between audio, text, and visual explanations, enabling intelligent multimodal understanding and gradual fading of assistance.

Gemini 2.5 is used for speech to text because of its stable and reliable audio transcription. Nano Banana is used to generate visual explanations when additional support is needed.

Challenges we ran into

Designing adaptive support without over helping the user Avoiding long term dependency while staying accessible Working within API limits and free tier constraints Simulating real world APD challenges without clinical datasets Managing full stack development and a Chrome extension under tight time constraints

One of the hardest challenges was deciding when the AI should help and when it should step back.

Accomplishments that we're proud of

Built an APD focused solution instead of a generic accessibility tool Designed behavior based adaptation instead of manual settings Integrated advanced Gemini reasoning models into a real working product Delivered a complete full stack prototype within limited time and resources

What we learned

Building AuraSync end to end taught us a lot.

We learned how to design and deploy a full stack AI application. We learned how to build browser based experiences such as Chrome extensions. We learned how to integrate and balance Gemini 2.5 and Gemini 3 models. We learned how to design adaptive AI systems that prioritize human growth. We learned how accessibility and AI intersect in real world applications.