Second Eyes

An editorial agent that turns hundreds of creative assets into a human-ready shortlist — checking the brief, filing obvious mismatches, and surfacing only what needs a second look.

Comment

Inspiration

I started with a practical problem: hundreds of old brand photographs, a publishing deadline, and a human who still needed to make the final creative choice.

The obvious AI product would rank the images.

Pick the best three. Score the rest. Automate the taste.

That was exactly what I did not want to build.

Creative judgment is often the part of the job worth preserving. The expensive part is everything around it: checking what is already known, noticing contradictions, finding missing evidence, and repeatedly reviewing material that never needed human attention in the first place.

So I started with a different question:

Could an agent become more useful by knowing when not to answer?

That became Second Eyes.

Second Eyes hands you a shortlist first. Most things stay filed. Questions come to you.

Instead of trying to automate taste, Second Eyes is designed to protect human attention for the cases that actually require judgment.

The hardest design problem was not classification.

It was uncertainty.

What it does

Second Eyes separates evidence handling from human judgment.

Its core metaphor is Drawer → Desk.

The Drawer holds cases the system can handle quietly. The Desk receives the cases where a person is actually needed.

Second Eyes has four routing states:

  • CLEAR — an existing comparable record and the new observation agree.
  • NEW — there is a valid new observation but no comparable prior claim. It is recorded as unverified, not promoted into truth.
  • CONFLICT — comparable evidence exists on both sides and disagrees. Both are preserved and the case goes to a human.
  • UNKNOWN — there is not enough comparable evidence to decide. The system refuses to guess and sends the question to a human.

Only CONFLICT and UNKNOWN enter the human-review queue. CLEAR and NEW stay quiet.

The principle underneath the product is simple:

Human judgment is not the failure state. It is part of the architecture.

And:

Unknown is a valid result.

Second Eyes does not rank heroes, score taste, or choose the final three photographs.

The human keeps that decision.

How we built it

Second Eyes is built around one architectural contract:

The model describes; only route() decides.

The implemented prototype uses the Strands Agents SDK for the agent/provider layer.

For the working synthetic image path, a fresh Strands Agent is created for each fixture and uses Gemini 2.5 Flash via Vertex AI as the observation provider.

The model returns observations from a frozen vocabulary.

It does not choose CLEAR, NEW, CONFLICT, or UNKNOWN.

Routing happens afterward in a deterministic pure Python function.

That separation is intentional: a persuasive model response cannot silently turn uncertainty into confidence.

The current implementation includes:

  • deterministic four-state route()
  • frozen vocabularies for comparable colour, item, and shot observations
  • mandatory evidence references
  • rejection of ref-less observations before routing
  • fresh-agent-per-fixture isolation
  • strict JSON parsing
  • validated JSONL run records
  • raw model-response and token-usage logging
  • Python and JavaScript regression suites covering routing, custody, replay, export privacy, packaging, and Sample Case interactions
  • 6 SHA-pinned synthetic fixtures
  • a static Publishing Workroom
  • a synthetic four-state viewer

The current repository also includes a reproducible six-image recorded runtime and an interactive Sample Case Mode. Visitors can set a bounded brief, inspect the computed Shortlist, Needs Review, Remaining, and coverage, keep final candidates, and download their decision. It uses custody-verified public derivatives and recorded observations; it does not upload visitor files or make a fresh model call. The recorded result is 1 Shortlist, 2 Needs Review, and 3 Remaining.

The synthetic runner verifies fixture hashes, sends image media to the observation provider, validates returned selections, attaches the declared evidence reference, and then passes that evidence to route().

The fixtures prove that the plumbing works.

They do not prove accuracy.

The 400-photo Workroom

The visual demo presents a publishing scenario based on 404 real source files, framed as a 400-photo deadline.

The Workroom shows:

  • SHORTLIST 18
  • NEEDS REVIEW 7
  • REMAINING 375

These counts and placements are deliberately illustrative mock content.

They are not outputs from a live 400-photo model run.

The Workroom demonstrates the intended interaction model: most material stays filed, while genuine questions reach the editor.

AWS and provider boundaries

Second Eyes uses the Strands Agents SDK.

I also configured an AWS Bedrock path and confirmed connectivity to Amazon Nova Lite and Nova Pro. A minimal inference attempt reached Bedrock but was stopped by the available daily token quota.

So I do not claim Nova model results.

The successful model evidence in the current repository comes from the Gemini/Vertex image path.

Video is explicitly unsupported in that implemented provider path and is logged as BLOCKED_UNSUPPORTED.

The public Workroom, recorded runtime, and Sample Case Mode are also mirrored as a static presentation layer on AWS Amplify Hosting. This deployment does not run backend inference; model observation remains a separately documented path.

That boundary matters.

A system designed to manage evidence should also be honest about its own evidence.

Challenges we ran into

Defining uncertainty

UNKNOWN sounds simple until you have to define it structurally.

If the model observes an item but the old record only contains a colour, both sides contain evidence — but nothing is actually comparable.

That must be UNKNOWN, not CLEAR.

If the model makes a valid observation and there is no comparable prior claim at all, that becomes NEW: record it, but do not pretend it is verified.

Those distinctions became code, not prompt instructions.

Preventing plausible claims from becoming facts

A legal vocabulary token is not enough.

If an observation says beige but has no evidence reference, Second Eyes discards it for routing purposes.

The system is not only asking:

“Does this answer look valid?”

It is asking:

“What evidence is attached to this claim?”

Separating observation from decision policy

It would have been easier to ask one model to inspect the image, compare evidence, assess confidence, and choose the route.

But then the most important product behavior would also become probabilistic.

Instead, the model observes.

Deterministic code decides.

Keeping the demo honest

A polished interface can accidentally imply capabilities the runtime does not have yet.

So the Workroom explicitly labels simulated counts and placements.

Presentation should never silently upgrade a prototype into a capability claim.

Accomplishments that we're proud of

The thing I am most proud of is not that Second Eyes can produce another AI answer.

It is that the system has places where an answer is not allowed.

The current synthetic image path can:

observe → attach evidence → validate → route deterministically → flag human review → log the run

The four routing states are tested rather than left as prompt semantics.

A missing evidence reference cannot quietly pass through.

A contradiction is preserved instead of overwritten.

A gap stays visible instead of being filled with something plausible.

And the repository draws a hard line between implemented behavior and presentation-only concepts.

That makes the prototype narrower.

I think that is a strength.

What we learned

The biggest lesson was that trust is not a model feature.

It is an architectural property.

You can ask a model to be cautious, but prompts alone cannot define what the system is allowed to claim.

That requires boundaries in code.

I also learned that abstention becomes much more useful when it is treated as a normal output rather than an error.

In Second Eyes, UNKNOWN does not produce a failure screen.

It produces a question.

And human-in-the-loop does not mean a person is there to rescue the AI.

For creative work, the human is not the fallback.

The human is the reason the system exists.

The machine should reduce noise around that person's judgment.

What's next

Second Eyes now includes a recorded, custody-verified six-image runtime and a public interactive Sample Case Mode. The runtime verifies the asset manifest and SHA lineage, recomputes evidence routing, applies a fixed publishing brief, fails closed when required signals are missing, and derives Shortlist, Needs Review, Remaining, and coverage from sealed results.

This is a recorded small-set run, not a live inference endpoint or an accuracy claim. Next, we want to test this workflow with external users before adding bounded uploads and fresh inference. That phase requires server-side credentials, explicit retention and deletion rules, private result access, and the same fail-closed evidence boundaries demonstrated here.

The current runtime is verified locally with full custody logging; the next step is a Bedrock AgentCore deployment for managed sessions and fresh inference. The agent is already shaped for it — one fresh Strands agent per photo, stateless decisions, validated run records — so the custody chain (SHA → evidence ref → route → brief-gate) transfers unchanged.

The long-term goal is not a machine that makes every decision.

It is a machine that earns the right not to interrupt you.

Most things should stay filed.

Questions should come to you.

And when the evidence runs out, the blank card should stay blank.

Second Eyes does not automate taste. It protects the human attention required for taste.

Share this project:

Updates

Submission history