KnowYourRights

• 

  Logo

Inspiration

My girlfriend is the one that really got me thinking about this problem long before. Her being hispanic has made the threat of ICE all too relevant in her life. Over 50 million immigrants live in the United States, with 2 million+ facing pending immigration court cases, most having no lawyer and no idea what to do. I wanted to build something that could be there for those people, on a platform people already use.

What It Does

KnowYourRights is a WhatsApp-integrated chatbot that delivers accurate, source-verified legal rights information in real time. Users can send text messages or photos of documents (warrants, notices) and receive concise, actionable guidance in English or Spanish — all backed by documentation provided by the American Civil Liberties Union (ACLU).

How We Built It

User (text/photo) → WhatsApp → Twilio API → Ngrok webhook → RAG Engine → ChromaDB + Gemini 3.0 → Source-verified response. We built a RAG engine that queries an ACLU knowledge base stored in ChromaDB as vector embeddings. Gemini 3.0 generates responses strictly grounded in retrieved documents, preventing hallucinations. The system supports conversation memory for personalized interactions and an urgent mode that delivers short, actionable answers for crisis scenarios.

Challenges

SMS to WhatsApp pivot: SMS verification timelines were too long for a hackathon. It would take days before I could get a phone number verified so I pivoted to WhatsApp, another commonly used messaging app via Twilio, then had to engineer workarounds for sandbox message limits. Gemini API quirks: Gemini has free-tier rate limits which need to be handled as to not send too many requests too quickly and model names in the Python library didn't match the Gemini AI Studio website. This led to a bit of confusion in the process of getting the initial MVP running. RAG design decisions: Balancing when to hardcode functionality into the RAG engine vs. letting the LLM classify inputs autonomously took significant iteration. Standing out: LLM chatbots already exist everywhere — my differentiator is the combination of WhatsApp accessibility and sourced responses.

What We Learned

You can't just build a chatbot on some random platform and expect people to use them. Building for marginalized communities means meeting them where they already are and making usage seamless — not asking them to download a new app or navigate to a website. I learned how to build a production-style RAG pipeline, integrate with the Twilio/WhatsApp API under tight constraints, and gained experience with Gemini and Google AI Studio.

What's Next

Voice and video recognition for hands-free use during crisis, additional languages (Mandarin, Arabic, Vietnamese), local legal aid integration to connect users with nearby immigration lawyers, and SMS fallback for users without internet access.

Built With

• beautifulsoup4
• chromadb
• computer
• flask
• gemini-api
• google-gemini-3.0-pro
• multimodal-ai
• natural-language-processing
• ngrok
• pdfplumber
• pypdf2
• python
• rag-(retrieval-augmented-generation)
• streamlit
• twilio-whatsapp-api
• vector-databases