PICTALK

• 

  generated by Gemini

PICTALK – Adaptive AAC with Gemini 3.0 Flash

PICTALK improves Augmentative and Alternative Communication (AAC) by replacing static menus with Dynamic Choice Sets. Our core design approach was inspired by the AACessTalk app. We studied how AACessTalk supports communication for speech-impaired users. We built on this foundation and extended it with real-time AI generation. Traditional AAC systems use fixed menus and slow navigation. PICTALK uses Gemini 3.0 Flash to generate communication cards in real time. This reduces the time needed for users with autism and speech impairments to express themselves.

Gemini 3.0 Flash: The Core Engine

PICTALK uses the native multimodal features of Gemini 3.0 Flash to improve speed and accuracy.

• Raw Audio Analysis: Gemini listens to ambient audio directly. It captures context and suggests the most relevant communication cards.
• Regenerate Feature: Users can regenerate cards for the same question. This provides new options and improves flexibility.
• Structured JSON Output: Gemini generates strict JSON output. It creates 12 optimized cards in Topic, Action, and Emotion categories. These cards appear instantly in the React interface.
• Real-time Contextual Reasoning: Gemini maintains conversation memory. Suggested cards change naturally as the conversation continues.

Built With

• gemini3
• google
• node.js
• notebooklm
• react
• typescript