Fallback

Inspiration

Most product reviews focus on the happy path: Can the user complete the task when the connection is stable, the data is available, and nothing interrupts them?

That leaves out the moments that often decide whether a user trusts the product.

What happens when a payment request times out after the user taps "Submit"? Can they safely retry, or will they be charged twice? What happens when someone switches apps halfway through a form? Does an empty dashboard explain what to do next, or does it simply look broken?

I built Fallback to review those overlooked paths. Instead of asking only, "Does this flow work?", Fallback asks, "Does it still work when real life gets in the way?"

What it does

Fallback reviews a product screen or flow for resilience problems.

A user can:

• Describe the flow they want reviewed
• Upload up to eight PNG, JPG, or WebP screenshots
• Paste a live URL for automatic desktop and mobile capture
• Select conditions such as unreliable networks, accessibility needs, first-time use, interrupted tasks, or empty data
• Add details about the primary action, technical stack, and the failure they are most worried about

Fallback then produces a prioritized report. Every issue includes:

• Severity and confidence
• The affected user
• The specific risk
• Evidence from the supplied material
• A recommended product or engineering change
• Suggested recovery copy
• A concrete acceptance test

The report also includes a resilience coverage score. Users can export it as Markdown, ask follow-up implementation questions, or share it through a persistent report link.

How I built it

Fallback is a full-stack Next.js 15 application built with React 19 and TypeScript.

I used Codex, powered by GPT-5.6 Terra, to scaffold the initial application structure, including the frontend and the three main API routes:

• POST /api/analyze creates a resilience review
• POST /api/capture captures a live page
• POST /api/chat answers follow-up questions about a completed review

Codex helped me move quickly through the initial setup and boilerplate. The main design work came afterward: shaping the review format, defining the failure conditions, grounding every finding in evidence, validating model output, and making the application remain usable when a provider is unavailable.

The analysis pipeline uses a separate model pass for each selected condition. For example, network resilience and accessibility are reviewed independently rather than being compressed into one broad prompt. These passes run concurrently and are merged into one report.

For now, Gemini-3.5-flash powers the review path, including screenshot vision and report-grounded chat. Fallback also supports OpenAI-compatible or Gemini-compatible gateways.

The model receives the screenshots and product description as untrusted reference material. The prompt tells it not to claim that it tested source code, APIs, accessibility trees, or real network behavior. It must return structured JSON, and the server validates every issue before including it in the final report.

For live URL reviews, Playwright opens the page in two views:

• A 1280 x 800 desktop view with a full-page capture
• A 390 x 844 mobile view with touch and mobile settings

Both pages load and capture concurrently. This matters because a product can behave well on desktop while hiding important controls or recovery states on mobile.

Reviews are stored in SQLite using better-sqlite3. The database stores the report and review metadata, but not the uploaded screenshot contents. Each saved review receives a stable ID that powers the read-only /report/[id] share page.

Graceful degradation

I wanted Fallback to be usable even without an API key, and I did not want provider failures to destroy the entire workflow.

The application supports three modes:

1. OpenAI-compatible API
2. Google Gemini
3. A built-in demo catalog

If no provider key is configured, Fallback runs in demo mode. If a live provider fails, returns an authentication error, produces invalid output, or returns no usable findings, the application falls back to the demo catalog instead of returning a broken page.

The report clearly labels demo output. It never presents fallback content as if it came from a successful live analysis.

Challenges I faced

Grounding the review without pretending to test the product

A screenshot can reveal missing feedback, weak empty states, unclear controls, or a lack of visible recovery options. It cannot prove how the backend handles retries or whether an element has the correct accessibility semantics.

The first challenge was making the model useful without allowing it to overstate what it knew. I addressed this by requiring an evidence field for every finding, adding confidence levels, and explicitly limiting the claims the model can make.

This changed the output from generic criticism into an auditable review. A team can see both the recommendation and the observation that led to it.

Getting reliable structured output

Model responses are not always clean JSON. They may include Markdown fences, introductory text, or extra reasoning. Some Gemini models can also return separate thinking parts.

I added several safeguards:

• JSON response modes where the provider supports them
• Extraction of the outermost JSON object
• Filtering of Gemini thinking parts
• Runtime validation of every issue
• Allowed-value checks for context, severity, and confidence
• Graceful fallback when no valid findings remain

The application does not trust the model response just because it looks correct.

Supporting screenshot vision across providers

Gemini and OpenAI expect images in different request formats. Gemini uses inline base64 data, while the OpenAI chat format uses image URL content blocks.

I built separate image conversion paths while keeping one shared review schema. This allows the rest of the application to remain provider-independent.

Capturing real websites

Many modern sites never reach Playwright's networkidle state because they use analytics, live feeds, background polling, or WebSockets. Waiting for perfect network silence caused captures to hang.

I changed the capture strategy to wait for the page's load event, fall back to domcontentloaded, wait for fonts, and then allow a short period for the interface to paint. I also use JPEG compression and fall back from a full-page image to a viewport image when a page is too large.

Keeping the product usable when the model is unavailable

External model calls can fail because of quotas, invalid keys, model changes, network errors, or malformed responses. Returning a generic server error would make Fallback itself fail the resilience test it is supposed to promote.

The demo catalog became part of the product architecture rather than a temporary placeholder. It provides a complete review flow with realistic issues, recommendations, recovery copy, and acceptance tests while clearly stating that the output is a demo.

What I learned

The biggest lesson was that resilience is not only an infrastructure concern. It is also a product design concern.

A backend can retry a request correctly while the interface still leaves the user unsure whether anything happened. A form can preserve its data but fail to tell the user that their progress was restored. An empty state can be technically correct and still feel like an error.

I also learned that confidence and evidence make AI feedback much more useful. A list of polished recommendations is easy to generate, but it is hard for a team to trust. Showing what the model observed, and how certain it is, gives the team something they can challenge and verify.

Using separate passes for each condition also produced better results than one large prompt. A focused accessibility pass notices different details from a network-recovery pass. Running those passes concurrently preserves that depth without making the user wait for them one at a time.

Finally, building graceful degradation changed how I thought about AI applications. The model should improve the product, but the entire product should not disappear when the model is unavailable.

What I am proud of

I am most proud that Fallback does not stop at identifying a problem.

A finding such as "the retry experience is unclear" is not enough. Fallback turns it into something a team can act on:

• Who is affected?
• What could go wrong?
• What evidence supports the concern?
• What should the team build?
• What should the interface say?
• How can the fix be tested?

That makes the report useful to designers, product managers, and engineers without requiring them to translate vague AI feedback into implementation work.

Fallback makes the overlooked path visible before a real user is forced to find it.

Built With

• openai