InspiInspiration
A photo is easy to look at and hard to search, share with a screen reader, or explain in one sentence. I wanted a small tool that does one honest job: look at a picture and say what is there, in plain language. That is useful for albums, websites, and anyone who cannot see the image.
What it does You drop in a photo or pick a sample. Gemini Lens returns:
A short caption A slightly longer story of the scene Alt text you can paste into a website Tags, plus simple notes about light and color How I built it I started the idea in Google Colab with the Hugging Face transformers library. The prototype is now a local Flask app:
BLIP (Salesforce/blip-image-captioning-base) reads the image and writes sentences One pass asks for a short caption; another asks for a more detailed reading Pillow measures size, brightness, warm/cool color, and dominant hues A simple web page handles upload, samples, and copy-to-clipboard No extra API key is required to try the prototype.
Challenges Running an image-to-text model on CPU is slower than a cloud API, so the first click takes a few seconds. I also had to keep the writing simple — a caption should sound like a person, not a research paper.
What's next A mobile-friendly share flow Stronger long descriptions for complex scenes Optional cloud models (for example Gemini) when an API key is available Export of a caption sheet for a whole folder of photosration
Gemini Lens turns a photo into plain language.
You upload an image or tap one of the sample photos. The app looks at the picture and gives you four useful outputs:
Short caption — one clear sentence, like “A dog running through the grass with a ball in its mouth.” Story — a longer, easy-to-read description of the scene Alt text — ready to paste into a website so screen readers can describe the image Tags and visual notes — keywords plus simple facts about size, light, warm/cool color, and main hues The goal is practical: help people understand, search, and share images even when they cannot see the photo, or when they need a sentence fast.
How we built it We started with a simple idea in Google Colab: use Hugging Face Transformers to caption images. That notebook became a real web prototype you can click through.
Pipeline
The browser sends the photo to a local Flask app. BLIP (Salesforce/blip-image-captioning-base) reads the pixels and writes a sentence. We ask the model more than once — once with no prompt for a short caption, and again with prompts like “a detailed photograph of” and “an image of.” Pillow studies the same photo for orientation, brightness, color temperature, edge detail, and dominant colors. Those facts are merged into a story, alt text, and tags. The UI lets you drag and drop, try sample images, and copy any result in one click. Stack: Python, Flask, PyTorch, Hugging Face Transformers, Salesforce BLIP, Pillow, HTML/CSS/JavaScript.
No paid API key is required to run the prototype. The model runs on the machine, so a photo stays in the app.
Challenges we ran into Moving from a notebook to an app. Colab is good for a first test. A hackathon demo needs a page someone else can use. We had to wrap the model in a small server, add upload and sample images, and show clear loading and error states.
Speed on CPU. Image-to-text models are heavy. Without a GPU, one photo can take several seconds. We load the model in the background, show a “Model ready” chip, and tell the user the app is reading the image instead of freezing with no feedback.
Captions that sound human. The raw model output can be short, repetitive, or oddly phrased. We clean punctuation, ask the model in more than one way, and add only facts we can measure (light, color, frame) so the longer text stays honest.
Keeping the idea simple. It was tempting to add too many features. We stayed with one job: look at a picture and say what is there, in language a person can use.
Accomplishments that we're proud of We shipped a working prototype, not only a slide or a notebook. A real photo goes in and a real sentence comes out. On the first test, Gemini Lens described a dog running with a ball in its mouth. The app is usable by someone who is not a programmer: drop a photo, press Read this image, copy the text. We produce more than a caption. Alt text, tags, and light/color notes make the project useful for accessibility and for organizing photos. The whole flow runs locally with an open model, so anyone can try it without signing up for an API. What we learned A good hackathon project is a small idea finished well. “Image to sentence” is easy to explain and easy to demo. Starting in Colab helped us prove the model quickly. Turning it into Flask taught us about file uploads, model loading, and user feedback. Vision-language models are strong at the main subject, but extra context (light, color, frame) still needs simple computer-vision checks. Accessibility is not an extra slide. If the output is clear alt text, the product already helps someone. Clear UI matters as much as the model. People will wait a few seconds if they understand what is happening. What's next for Gemini Lens Stronger long descriptions for busy scenes, signs, and photos with several people Optional cloud models (including Gemini) when an API key is available, for richer detail Folder mode — caption many photos at once and export a sheet for an album or website Mobile-friendly share flow so someone can describe a photo from a phone Language options, starting with simple English and later Hindi, so more people can use the captions Better accessibility — test alt text with screen readers and improve the wording from real feedback
Log in or sign up for Devpost to join the conversation.