QueensLingo

Logo

Inspiration

This project was inspired by a moment that every immigrant child has lived. A parent hands them a paper, could be a legal notice, an electricity bill, a registration form, and says the infamous line "what does this mean?" The child is often times very early in their educational career and have limited knowledge themselves. They are tasked with translating complex words, and dense grammatical structure that they don't fully understand themselves. These children or tasked to carry a weight that no child should carry. We built QueensLingo to help these parents and lift the weight of the shoulders of these children.

What it does

QueensLingo lets users point their phone camera at an official document and instantly understand it. The app captures the image, uses Gemini AI to extract and interpret the document, and returns a plain-language explanation in the user's NATIVE language — not just translated, but contextualized. It tells you what the document is, what it's asking, why it matters, and what happens if you ignore it. It then READS that explanation ALOUD in their native language using ElevenLabs multilingual text-to-speech. Finally, it surfaces real nearby organizations — legal aid offices, immigrant services, housing courts — pulled dynamically from Google Places based on the zipcode provided.

How we built it

The tech stack consisted of: Next.js, TypeScript, Tailwind CSS, Shadcn, Framer Motion for the frontend, and Google Gemini 2.5 Flash, ElevenLabs API, Google Maps + Places API for the backend. The application is deployed via Vercel and is optimized for mobile use due to the convenience mobile devices provide when using the camera feature (utilizing react-webcam).

Challenges we ran into

The two main challenges we ran into was getting ElevenLabs to return the full text-to-speech file. We are using a free account with a limited number of credits and many features were limited. Due to this we were unfortunately able to support text-to-speech for 16 languages, and couldn't perform too much testing as we did not want to exhaust too many of our credits prior to demo'ing.

Accomplishments that we're proud of

We are extremely proud of how the workflow is for the AI pipeline. Since many of our core features rely on one singular event (document being captured) we were worried the application performance would be slow but we were able to nest the calls in such a way that as soon we got responses from our calls we were able to start presenting information to the user immediately.

What we learned

We learned how to approach hackathon builds the right way. In prior hackathons we attended we focused too much on minute details such as the landing page, branding, and user authentication options. This gave us less time to polish our actual tool we wanted to provide for our user. This time around we knew to build out our intended functionality first and work backwards to have the best product to present.

What's next for QueensLingo

QueensLingo has a lot of room for growth. First and foremost would be setting up user authentication and a database to provide a better long lasting user experience. This would allow users to save the details regarding their previously scanned documents without the need to rescan the document. We also would benefit greatly from upgrading the ElevenLabs model we are using to provide text-to-speech functionality for more languages and a voice that is better tailored to handle enunciating in these languages.