Inspiration
Alexa is already good at answering questions. But many real-world questions do not have an answer available the moment they are asked.
Why is my Wi-Fi slow every evening?
Why does my refrigerator sometimes stop cooling?
Why does a battery drain unusually fast?
Why does a plant keep wilting even though it is being watered?
For questions like these, another plausible explanation is not enough. What is useful is a way to investigate.
That became the idea behind WhyLab:
Alexa should not just answer "why?" It should help you find out.
WhyLab is a persistent scientific reasoning agent for real-world causal questions. Instead of immediately choosing one explanation, it keeps multiple hypotheses alive, identifies what evidence could support or contradict them, remembers observations, and recommends what to test next.
The goal is to move conversational AI from generating explanations toward helping users discover which explanation actually survives contact with evidence.
What it does
A user starts with an open-ended causal question such as:
My Wi-Fi gets slow every evening even though it works well during the day. Help me figure out why.
WhyLab turns that question into a structured investigation with measurable variables, competing hypotheses, predictions, falsifiers, uncertainties, and explicit evidence rules.
Every hypothesis begins OPEN.
That is intentional. WhyLab does not treat a fluent explanation as evidence.
When the user records an observation, WhyLab evaluates it against explicit evidence rules. A hypothesis can then become SUPPORTED or CONTRADICTED, while hypotheses untouched by that evidence remain OPEN.
WhyLab also keeps an evidence trail showing what changed and why.
Once evidence starts narrowing the possibilities, a planner recommends the next discriminating test. The goal is not simply to collect more information. It is to choose a measurement that can distinguish between the explanations that are still plausible.
For example, in one live production investigation, WhyLab generated four explanations for evening Wi-Fi slowdown. After observing very low local network load, the household-congestion hypothesis became contradicted, while interference, router overheating, and ISP throttling remained open.
The core loop is:
question → competing hypotheses → observation → evidence update → next falsifiable test → revised investigation
Investigations persist, so the reasoning process can continue across time instead of disappearing after one conversation.
This interaction model is especially suited to Alexa+. Real investigations often happen in the physical world and unfold over minutes, hours, or days. A user should be able to ask a question where the problem occurs, perform a check, return with an observation, and continue from the same investigation.
How we built it
The central architectural decision in WhyLab is that generating a possible explanation and deciding whether that explanation is supported are different jobs.
WhyLab uses an LLM at runtime through OpenRouter to interpret previously unseen causal questions. The model helps generate the initial structured investigation, including hypotheses, measurable variables, predictions, falsifiers, uncertainties, and candidate evidence rules.
Those outputs are validated before they become part of the investigation.
After that, the LLM is not allowed to decide whether its own hypothesis is true.
Evidence evaluation is deterministic.
When an observation arrives, WhyLab compares it against explicit evidence predicates. Those rules determine whether a hypothesis remains OPEN, becomes SUPPORTED, or becomes CONTRADICTED.
A separate planner then examines the unresolved hypotheses and recommends the next useful measurement, prioritizing tests that are specific, discriminating, and capable of falsifying an explanation.
WhyLab is built with:
- Python
- FastAPI
- Pydantic
- SQLite
- Model Context Protocol
- React
- Vite
- Modal
- OpenRouter
The application and MCP server are deployed together on Modal, with persistent storage for investigations.
WhyLab exposes four MCP tools for the reasoning workflow, including starting an investigation, recording evidence, retrieving the evidence report, and requesting the next test. The production MCP service runs over Streamable HTTP and currently negotiates MCP protocol version 2026-07-28.
I also built an Alexa+ concept experience in the web interface. It is grounded directly in the live WhyLab investigation state and demonstrates how the same reasoning loop could work conversationally through Alexa+, while clearly distinguishing the concept interface from native Alexa+ runtime access.
The deployed system has been tested end-to-end, and the current automated test suite contains 63 passing tests.
Challenges we ran into
One of the hardest design problems was deciding how much authority to give the generative model.
WhyLab needs generative reasoning because users can ask previously unseen questions across different domains. Hard-coding every possible hypothesis would defeat the purpose of the system.
But allowing the same model to generate a hypothesis and later decide whether that hypothesis was supported would make the reasoning circular and difficult to audit.
The solution was to separate generation from evaluation. The LLM can propose possibilities, but it cannot assign their epistemic state. Once an investigation exists, observations are evaluated against explicit rules and deterministic logic changes the state.
A second challenge was generation reliability.
Free model routes sometimes returned incomplete or malformed structured output. During production testing I encountered missing tool calls, unexpected fields, and hypotheses without both supporting and contradicting evidence rules.
Rather than hiding these failures, I reproduced them in regression tests and hardened the interpreter to validate, normalize, retry, or safely reject unusable generations without inventing evidence.
Generalization created another challenge.
The earliest working version used a basil-wilting example, and some early smoke tests accidentally encoded assumptions from that domain. I replaced those assumptions with tests derived from each investigation's actual evidence rules.
That allowed the same reasoning loop to work on unrelated questions involving networking, refrigerators, laptop batteries, and website performance.
Accomplishments that we're proud of
The accomplishment I am most proud of is that WhyLab is not a scripted demonstration around one example.
The production system accepts previously unseen causal questions and creates new investigations from them.
It can:
- generate competing hypotheses
- persist investigations
- record real observations
- update hypothesis states deterministically
- maintain an auditable evidence trail
- recommend the next discriminating test
- expose the workflow through a live MCP server
The deployed MCP workflow has also been tested end-to-end against the public production service.
I am equally proud of something less visible: WhyLab is designed to say,
"We do not know yet."
That is treated as a valid state, not a failure.
A hypothesis remains OPEN until evidence actually changes it. WhyLab does not convert model confidence into apparent knowledge.
That distinction is central to the project.
What we learned
Building WhyLab changed how I think about conversational agents.
For many real-world questions, the most useful agent is not the one that gives the fastest or most confident answer.
It is the one that can represent several plausible explanations at once, remember what has already been observed, distinguish assumptions from evidence, revise conclusions when evidence changes, and identify what would be most useful to test next.
I also learned that tool use alone does not make an agent scientifically useful.
The deeper design problem is deciding which parts of reasoning should remain generative and probabilistic, and which parts should be explicit, deterministic, and auditable.
MCP was particularly useful because it lets the reasoning engine remain independent of the interface. The same persistent investigation state can be exposed to different conversational surfaces without moving the scientific logic into the frontend.
The project also reinforced the importance of testing the failure modes of generative systems rather than only testing their successful outputs. Several of WhyLab's most important reliability improvements came directly from reproducing failures observed in the deployed system.
What's next for WhyLab
The next step is deeper integration with native Alexa+ capabilities as broader developer access becomes available.
I would also extend WhyLab with:
- scheduled follow-ups for investigations that unfold over time
- sensor and device-derived observations
- richer evidence provenance
- collaborative investigations
- domain-specific measurement adapters
- stronger calibration of generated evidence thresholds
- systematic evaluation of whether proposed tests actually reduce uncertainty
I also want to develop a benchmark around a question that matters beyond this hackathon:
Can an AI system identify the next observation that is genuinely useful for distinguishing between competing causal explanations?
The long-term goal of WhyLab is not to make AI sound more certain.
It is to give AI a better process for finding out.
Log in or sign up for Devpost to join the conversation.