CivicConnect

Inspiration

We realized that millions of citizens, especially those who are deaf or hard of hearing, struggle to access government services and documents because most digital interfaces rely only on text or speech. We wanted to create an inclusive AI assistant that makes every civic document speak, sign, and explain itself, giving equal access to all, regardless of communication mode or ability.

What it does

CivicConnect is an accessibility-first civic assistant that: Fetches and summarizes government documents (forms, policies, FAQs) using RAG and Gemini. Converts responses into spoken explanations using ElevenLabs voice synthesis. Generates ASL animations from gloss text using our ASL 2D motion renderer. Displays all outputs (text, speech, sign) side-by-side for inclusive and empathetic accessibility. Users can ask:

“How do I apply for unemployment benefits?” and CivicConnect will find the official document, summarize it, explain it out loud, and sign it in ASL.

How we built it

Frontend: Built in Streamlit, designed for accessibility and clean civic UI.

Backend:

Gemini API → for multi-modal understanding and text generation. Python + FastAPI → to orchestrate responses and generate ASL glosses. Matplotlib + JSON motion files → to animate 2D ASL keypoints into videos. ElevenLabs API → for lifelike speech narration. Hosting: Deployed on Render

Challenges we ran into

Finding reliable ASL gloss-to-motion data for generating signs dynamically. Managing multi-modal output synchronization between text, voice, and animation. Integrating Chroma vector search for government data RAG efficiently. Designing an inclusive yet modern UI in a short time frame.

Accomplishments that we're proud of

Built a multi-modal AI accessibility tool within 25 hours. Seamlessly integrated LLM (Gemini) + speech & sign generation. Designed a visually beautiful Streamlit prototype for inclusive design. Created a working ASL animation engine that visualizes gloss sequences.

What we learned

How to leverage ChromaDB for RAG pipelines efficiently. Designing for accessibility first changes the way you think about UX. Converting text to ASL glosses and motion JSONs requires linguistic insight. Multi-modal LLM applications can be powerful bridges for inclusion.

What's next for CivicConnect

Integrate real ASL 3D motion capture data for lifelike signing. Add speech-to-sign translation for real-time interactions. Expand coverage to more government agencies and languages. Form auto-complete Suggestion

Built With

• elevenlabs
• fastapi
• gemini
• langchain
• streamlit