Community Evidence Review

Helps founders judge whether community evidence matters to their problem. AI keeps evidence, observations, inferences, and unknowns separate; the human makes the final call.

Comment

Inspiration

Founders often turn to online communities to learn whether a problem is worth exploring or building for an MVP. But direct evidence, limited observations, assumptions, and interpretations easily blur together, and asking AI for a verdict can hide what the evidence doesn't show.

Many tools focus on finding and scoring signals in online communities. Community Evidence Review explores a different role for AI in this use case which is not to decide for the founder, but help the founder organize supplied evidence and make uncertainty visible. Early research decisions are often made on thin evidence, and keeping evidence, interpretation, and uncertainty visibly apart helps people judge whether a problem is worth pursuing with the gaps in view.

The project follows one core principle based on having AI organize evidence and uncertainty and a human will make the final judgment based on the source.

What it does

Community Evidence Review is an AI-assisted working prototype designed and developed by Lashachi Inc. It helps founders and researchers examine how supplied evidence from one public online community relates to a research problem they might solve.

A user provides a specific research problem, identifies one public community, and supplies source items for review. The prototype does not search the web or retrieve content from the community. Because the supplied text is analyzed with OpenAI's API, it asks for public, non-sensitive material only.

Instead of blending everything into one conclusion, the results keep different kinds of information separate:

  • Direct Source Evidence: information directly supported by a supplied source, including published community rules, which are labeled as not independently verified
  • Limited Observations: what can reasonably be observed without going beyond the evidence
  • AI Inference: interpretations, tagged separately from established facts
  • Unknown: important questions the supplied evidence cannot answer

Findings maintain references to the supplied source items. Relationship labels such as Supports, Weakens, Complicates, and Context, describe how evidence relates to the research problem. In the demo, a missed watering on busy workdays supports the problem, while a kitchen timer that works fine weakens it.

These labels are not scores or verdicts. The prototype deliberately avoids numerical scores, confidence ratings, automated suitability decisions, and recommendations.

After reviewing the evidence and uncertainty, the user records their own judgment: investigate further, not enough evidence yet, or set aside. That judgment stays separate from the AI analysis and is not sent to OpenAI.

Results can be downloaded as a plain-text file. The prototype does not maintain a saved-review history, so users should download results they want to keep before leaving the page.

How we built it

The working prototype is built with: Next.js, React, TypeScript, Zod, and the OpenAI JavaScript/TypeScript SDK.

The browser sends the research material to a server-side workflow. The server validates the input and sends the supplied research context and source items to the OpenAI Responses API using gpt-5.6-luna with low reasoning effort. The response uses Structured Outputs, so the AI analysis follows a defined schema before it is validated and returned to the browser.

The OpenAI API key stays server-side and is not exposed to the browser.

The architecture is intentionally narrow. It has no database, user authentication, analytics, web search, live community retrieval, or additional AI provider. The focus is the evidence-review workflow itself.

The demonstration uses a fictional balcony-gardening research problem and five fictional source items, so the complete workflow can be shown without real community or personal information.

Challenges we ran into

The main challenge was deciding what AI model to use and designing what the AI should not do with community evidence.

It would have been easy to turn the output into a score, confidence percentage, recommendation, or simple "good fit / bad fit" answer. Instead, the prototype preserves the distinctions between source evidence, observations, AI inference, and genuine unknowns.

Another challenge was keeping the human decision structurally separate from the AI analysis. The final judgment belongs to the user, not the model.

The project also needed careful handling of failure and validation paths, so invalid requests don't unnecessarily reach the OpenAI API and AI responses must match the expected structure before they are displayed.

Accomplishments that we're proud of

  • Completed an end-to-end workflow in a deliberately small working prototype: a research problem and source items go in, a structured review comes out, the user records a judgment, and the results can be downloaded.
  • Human judgment that is kept structurally separate from the AI analysis and is not sent to OpenAI.
  • AI output that must match a defined schema and pass server-side validation before it reaches the user.
  • Restraint in the product design: no scores, no recommendations, and unknowns treated as a first-class part of the output.

What we learned

Adding AI to a workflow doesn't require handing the whole decision to the model. AI can organize information, identify relationships, and expose uncertainty while preserving meaningful human control. Structured outputs and application-level validation also place technical boundaries around how model output is used.

The project turned "human in the loop" into a concrete product design: the AI performs a defined analytical task, while the consequential judgment remains explicitly human.

What's next for Community Evidence Review

Community Evidence Review is currently a focused working prototype. Next steps include testing the evidence model on additional research scenarios and learning from real users whether the distinctions between evidence, observation, inference, and unknowns stay understandable and useful.

Any expansion or future improvements would preserve the central principle that AI organizes evidence and uncertainty and the human makes the final judgment.

Built With

Share this project:

Updates

Submission history