PickyLens

• 

  PickyLens

• 

Inspiration

AI is getting remarkably good at producing polished answers. That creates a new problem: a response can look excellent while still missing part of the assignment, misunderstanding a priority, contradicting itself, or answering something slightly different from what the user actually intended.

Many existing tools help improve prompts. Others evaluate AI systems for developers. I wanted to focus on the person doing real work with AI.

The question behind PickyLens became:

Did the AI actually deliver what you asked for — and if not, can you fix only what is missing?

That led me to think of PickyLens as AI quality control for people who have real work to do.

What PickyLens does

PickyLens supervises the interaction end to end.

Before the request is sent, it reviews what the user is asking and identifies material ambiguities, missing priorities, or unnecessary context. The user stays in control and explicitly approves the intent.

After the AI answers, PickyLens checks the response against that approved intent and classifies each requirement as Complete, Partial, Missing, or Needs review, with supporting evidence.

If something is genuinely missing, PickyLens does not ask the AI to repeat the whole task. It generates a targeted follow-up to:

Fix only what’s missing.

Then it can verify the combined result.

If the original answer is already complete, PickyLens simply says so and gets out of the way.

How I built it

I used the Devpost Learn Skill Pack as the actual development workflow, not just as documentation.

Using flipped interaction, my Codex coding agent interviewed me about the product, challenged assumptions, and helped turn my decisions into Scope, PRD and Spec before implementation began.

I remained responsible for the product decisions; the interview process helped expose questions and tradeoffs I had not fully considered.

From there, I followed:

Start → Scope → PRD → Spec → Build → Ship

The planning documents became working project memory and guided implementation in small, reviewed slices.

The standalone Web application is built with Python, FastAPI, Jinja2, vanilla JavaScript, SQLite, and the OpenAI Responses API.

I designed PickyLens around a provider-neutral core so the quality-control logic is separate from any particular AI host.

The Web interface evolved into a two-sided workbench:

• What I asked
• What I got

This made the interaction much easier to understand than exposing internal terms such as PRE, POST, NEXT, and VERIFY.

I also built a Chrome/Edge extension for Gemini Chat. It lets the user keep working inside normal Gemini, review a request before sending it, and check the resulting answer afterward.

Gemini generates the main response through the user's own account, while PickyLens performs an independent quality review through its backend.

Challenges I faced

One of the biggest lessons was that AI quality control must be honest.

During early testing, I expected to capture an AI response with a missing requirement for a repeatable demo. Instead, several live responses were complete. I stopped trying to manufacture a convenient failure.

Then I found something more interesting: one genuine response contained conflicting cost figures. PickyLens initially missed the contradiction.

Rather than hard-code that example, I improved the evaluator generically so it could detect material inconsistencies. The same captured response then became a real regression case.

I also learned that integrations must be validated on the exact surface users will use. Early experiments with other Gemini surfaces led me back to the real target: ordinary Gemini Chat in the browser. I tested that surface before committing to the final integration.

Automatic detection of exactly when a Gemini response finishes proved too dependent on undocumented DOM behavior. I chose a more reliable interaction instead: an explicit Check answer control beside the response.

That tradeoff made the product simpler and more robust.

What I learned

The most important lesson was that PickyLens should not need AI to fail in order to justify its existence.

Sometimes its most valuable answer is:

Everything requested is covered. No repair needed.

I also learned that AI supervision has its own computational cost. That did not constrain development, but it became an important product-design consideration.

The architecture therefore supports bounded usage, rate and token limits, and future options such as cheaper-model routing, credits, BYOK, and host-native inference where appropriate.

Where it ended up

PickyLens now runs as a live standalone Web application and as a browser extension inside normal Gemini Chat.

It includes request review, evidence-based answer checking, targeted recovery, verification, persistent interaction history, server-side usage controls, and 104 automated tests.

Most importantly, the product became clearer as I built it:

PickyLens checks whether the AI actually delivered what you asked for — and fixes only what’s missing.

I started by trying to improve prompts.

I ended by building a quality-control layer for AI work.

Built With

• chatgpt
• chromeextensions
• codex
• devpost-learn-skill-pack
• fastapi
• gemini
• javascript
• openaiapi
• python
• sqlite