-
-
ProblemForge | How the Reasoning Engine Works
Inspiration
AI can now give us answers almost instantly. That is powerful, but it also creates a new risk. We can get a very good answer to the wrong problem.
Imagine someone saying, “We need an AI app to help students manage overwhelming coursework.” The conversation has already jumped to a solution. But what if students are not overwhelmed because they need better planning? What if major assignments and exams from different courses keep landing in the same week? In that case, building a better study planner might not solve the real problem at all.
That is what inspired ProblemForge.
I wanted to build something that does not immediately ask, “What should we build?” It first asks questions like “What is actually happening?” “What do we know?” “What are we assuming?” “What else could be causing this?” and “What would change our decision?”
The idea is simple. Before spending time, money, and effort building a solution, make sure you understand the problem well enough to know what is actually worth testing.
That became the idea behind ProblemForge.
Find the right problem before you build the wrong solution.
What it does
ProblemForge turns a messy problem into a structured six-step journey.
Describe → Understand → Question → Reframe → Test → Plan
A user starts by describing a real problem in their own words. It can be incomplete, uncertain, or even contain a solution already. If someone writes, “We need an AI app to help students manage overwhelming coursework,” ProblemForge notices that “AI app” is already a proposed solution. It keeps that idea as an option, but it does not assume it is the right answer.
Next, ProblemForge breaks the situation apart. It identifies possible causes, the people involved, important limits, unanswered questions, assumptions, and the evidence already available.
It also makes uncertainty visible. Claims can be marked as Known, Inferred, Assumed, Needs Evidence, or Unknown. This matters because a possible explanation should not quietly become a fact just because it sounds convincing.
Then ProblemForge looks for an assumption that could actually change the decision.
In the guided example, the original thinking assumes that student overload is mainly caused by poor individual planning. Under that understanding, an AI study planner ranks #1.
ProblemForge challenges that assumption.
What if the problem is not mainly individual planning? What if high-stakes deadlines from different courses are piling up in the same weeks?
ProblemForge reframes the problem around that possibility and recomputes the same solution options.
The result changes significantly. Cross-course deadline coordination moves from #3 to #1, while the AI study planner falls from #1 to #4.
The solutions did not change.
The understanding of the problem changed.
But ProblemForge still does not say that the new #1 direction is automatically correct. It asks what should be tested before committing to it.
In this example, the next step is simple. Collect assignment and exam dates from one program and check whether high-stakes deadlines really do cluster in the same weeks.
Finally, ProblemForge creates an action plan that keeps the assumptions, evidence, unanswered questions, and next test visible.
The goal is not to make decisions for people. It is to help people make better-informed decisions before committing to a solution.
How I built it
I built ProblemForge as a full-stack application using Next.js, TypeScript, OpenAI, Zod, and Vercel.
One of my most important technical decisions was deciding what the AI should not control.
I did not want to send a problem to an AI model and simply ask it to produce an answer, some numbers, and a recommendation. That would be easier to build, but those numbers could look more trustworthy than they really are.
Instead, ProblemForge separates the work between the AI and a decision engine I built.
The AI handles the parts where language and exploration are useful. It can suggest possible causes, challenge assumptions, explore different ways of looking at the problem, and propose possible solution directions.
My own code handles the numerical decision logic. The AI does not directly generate the final numerical scores. The application calculates the scores, rankings, ranking changes, and which assumptions could change the result.
The approach comes down to one line.
The model proposes. The engine disposes.
I also give the model a defined structure for its responses instead of letting it return anything it wants and then trying to make sense of it afterward.
The same idea applies to evidence. ProblemForge keeps track of whether something is known, inferred, assumed, needs evidence, or is still unknown. Those labels are part of the reasoning system, not just words added to make an AI response sound careful.
I built both a live AI mode for analyzing new problems and a clearly labeled guided example that demonstrates the complete reasoning journey consistently.
Testing was also an important part of the build. The current version is backed by 287 automated tests and 12 reasoning invariants that check both normal application behavior and important rules in the reasoning system.
The result combines the strengths of both approaches.
AI helps explore the problem. Code keeps the decision process structured and consistent.
Challenges I ran into
One of my biggest challenges was figuring out how much control to give the AI.
It would have been much easier to let the model analyze everything, create the scores, rank the solutions, and tell the user what to do. But I realized that would go against the problem I was trying to solve.
If ProblemForge is supposed to challenge unsupported assumptions, its own results should not depend on numerical scores that an AI simply created.
That realization led me to separate AI reasoning from deterministic scoring.
Another challenge was keeping facts, assumptions, and possible explanations separate.
For example, “Students report feeling overwhelmed” and “Students are overwhelmed because they plan badly” are very different statements. The first may be something we know. The second is a possible explanation that still needs evidence.
AI-generated writing can easily make both statements sound equally certain. I had to design ProblemForge so that the difference remains visible throughout the experience.
I also had to make reframing meaningful.
I did not want ProblemForge to simply rewrite a problem using different words. If the understanding of the problem changes, that should affect what happens next. That is why ProblemForge can recompute the same solution options under a different working understanding and show whether the ranking changes.
Another challenge was avoiding a new kind of overconfidence.
If ProblemForge simply replaced the user's first answer with its own “better” answer, I would have recreated the same problem I was trying to fix. That is why even the highest-ranked direction is treated as something to test next, not something that has been proven correct.
Finally, I had to make all of this easy to understand.
Assumptions, competing causes, evidence, reframing, scoring, and testing can become complicated very quickly. I spent a lot of time turning those ideas into a six-step experience that someone can follow without needing to understand the theory behind it.
Accomplishments that I am proud of
The part I am most proud of is that ProblemForge does not just tell people to challenge their assumptions. It shows what can happen when they actually do.
The guided example demonstrates this clearly.
It starts with students feeling overwhelmed and an AI study planner already in mind as the solution. Under the first understanding of the problem, the AI study planner scores 7.75 and ranks #1. Cross-course deadline coordination scores 3.25 and ranks #3.
ProblemForge then identifies an important assumption.
“Student overload is primarily caused by poor individual planning.”
Instead of accepting that assumption, ProblemForge challenges it.
What if students are overwhelmed because high-stakes deadlines from different courses repeatedly overlap?
The problem is reframed around reducing those avoidable workload spikes, and the same solutions are scored again.
Now cross-course deadline coordination scores 7.75 and moves from #3 to #1. The AI study planner drops to 1 point and moves from #1 to #4.
That moment captures what ProblemForge is about.
Same solutions. New framing. Different leader.
But I am just as proud of what happens next.
ProblemForge does not tell the user to immediately implement cross-course deadline coordination. Instead, it proposes a small test. Collect assignment and exam dates for one program and find out whether deadline clustering actually exists.
That is important because ProblemForge is not supposed to replace one confident answer with another.
It turns uncertainty into something you can test.
I am also proud of the engineering behind the experience. The model does not generate the final numerical scores. Evidence status stays visible. Rankings are calculated by code. Unverified claims remain unverified. The current build is backed by 287 automated tests and 12 reasoning invariants.
What I learned
The biggest thing I learned is that building a useful AI product is not always about giving AI more control.
Sometimes the better question is what is AI good at, and what should normal software still handle?
AI is very useful for exploring possible causes, challenging assumptions, suggesting different ways to look at a problem, and generating possible directions.
But when I needed consistent scores and rankings, it made more sense to put that responsibility in code.
I also learned that showing uncertainty can make a product more useful, not less useful.
If ProblemForge shows that something is Assumed instead of Known, that is useful information. If it shows that one untested belief could change which solution ranks first, the user now knows where investigation could be especially valuable.
I also learned that reframing matters most when it changes what you do next.
Simply rewriting a problem is not enough. In ProblemForge, a different understanding can change the ranking of the same solutions and therefore change which direction deserves to be tested first.
Another lesson came from building ProblemForge itself. I had to keep questioning my own assumptions about how the product should work, what the AI should control, what the interface should claim, and how much confidence the system should show.
In a way, I had to use the ProblemForge mindset while building ProblemForge.
Most importantly, I learned that AI does not always have to be designed to answer faster.
Sometimes its job can be to help us think better before we act.
What's next for ProblemForge
The current version of ProblemForge takes someone from a messy problem to a structured action plan. The next step is to make that process continue after the first analysis.
Today, ProblemForge can tell users what evidence would be useful to collect. In the future, I want them to be able to bring that evidence back into ProblemForge.
For example, suppose a team investigates whether student deadlines really cluster. They should be able to enter what they found, update what is now known, and let ProblemForge reconsider the problem using that new evidence.
That would create a continuing cycle.
Understand → Question → Test → Learn → Update → Decide
I also want ProblemForge to become collaborative because real problems are rarely solved by one person.
A future team workspace could let people work on the same problem, disagree openly about assumptions, add evidence, compare different ways of framing the situation, and see which disagreements could actually change the decision.
Another important next step is connecting evidence directly to claims. Users could attach research, documents, interviews, survey results, or real data so that someone can see not only whether a claim is marked Known or Assumed, but also why.
I also want to explore ProblemForge across different kinds of problems. The same approach could be useful for startups, product development, education, business operations, nonprofits, public problems, and other situations where people may jump from a problem directly to a solution.
Eventually, I want ProblemForge to preserve the full story behind a decision.
What did the team believe at the beginning? Which assumptions were challenged? What did they test? What evidence did they find? Why did the ranking change? And why did they make the final decision?
That history could be just as valuable as the final answer because it shows how the decision was reached, not just what the decision was.
The long-term goal is not to build another AI tool that simply gives people more answers.
It is to help people understand before they decide, test before they commit, and learn before they scale.
As AI makes answers easier to generate, one question becomes even more important.
Are we solving the right problem?
Log in or sign up for Devpost to join the conversation.