
Architecture Decision Records That Survive Change

AI Evaluation Beyond Accuracy: A Practical Quality Framework

A product team can deliver a feature on time, meet every technical requirement, and still fail to improve the experience it was supposed to address. The code may be reliable, the interface may be polished, and stakeholders may be satisfied with the release. Yet the underlying difficulty remains because the team solved the problem it was asked to solve rather than investigating the problem users actually had.
This is why I consider problem framing one of the most consequential activities in digital product development. Before a team debates interfaces, features, roadmaps, or delivery estimates, it needs to understand what is happening, who is affected, why the situation matters, and whether the proposed explanation is supported by evidence. Otherwise, product discovery can become an elaborate process for validating a solution that was effectively selected before the research began.
The distinction matters because a feature request is not automatically a user need. A stakeholder may request a dashboard when the real difficulty is unclear responsibility. A customer may ask for notifications when the underlying problem is an unpredictable process. A product manager may propose automation when the workflow itself contains unnecessary steps. Each request may describe a plausible response, but none establishes that the team has identified the right problem.
My position is that effective problem framing should do more than produce an attractive problem statement. It should expose competing explanations, assess the strength of available evidence, separate stakeholder assumptions from observed user behavior, and establish conditions for rejecting an incorrect problem definition before substantial development resources are committed.
In this article, I will connect established product discovery and user research methods with a practical framework I call the Problem Framing Evidence Gate. The objective is not to introduce another bureaucratic stage into product development. It is to make the reasoning behind product decisions visible, challengeable, and proportionate to the risk of solving the wrong problem.
Decision Brief: What Should Problem Framing Establish?
The decision: Determine which user problem deserves investigation before committing to a proposed product feature or solution direction.
The recommended approach: Separate the requested solution from the assumed problem, develop competing explanations, evaluate the quality of supporting evidence, and identify what would disprove the leading interpretation.
The expected value: Product teams can make more defensible discovery decisions, identify unnecessary development commitments, and focus research on uncertainties that materially affect the product’s direction.
The limitation: Problem framing cannot guarantee product success or eliminate uncertainty. A well-supported problem definition still requires solution validation, feasibility assessment, business judgment, and continued learning after release.
What Is Problem Framing in Product Discovery?
Problem framing is the process of defining and examining a problem before committing to how it should be solved. In product discovery, it means identifying the people affected, the outcome they are trying to achieve, the obstacles they encounter, and the evidence that supports a particular interpretation of their difficulty.
It is important to distinguish problem framing from merely writing a problem statement. A problem statement is a useful artifact: it communicates a focused description of the situation that the team intends to investigate. Problem framing is the reasoning process that leads to that description, challenges its assumptions, and determines whether the description remains appropriate as new information becomes available.
The Nielsen Norman Group’s guidance on problem statements emphasizes identifying who is affected, what difficulty exists, and why it matters. It also warns against beginning discovery by investigating a selected solution rather than the underlying problem. This distinction is essential because a problem statement containing the answer can prematurely narrow the team’s understanding of the situation.
The Design Council’s Double Diamond framework similarly separates discovering and defining a problem from developing and delivering possible solutions. Its Discover and Define stages encourage teams to examine an issue broadly before narrowing their attention to a particular challenge.
However, I would not interpret this as a requirement to stop all solution thinking until research is complete. Early sketches, technical investigations, and disposable prototypes can help expose incorrect assumptions. The important difference is whether a proposed solution is treated as a hypothesis to investigate or as a commitment that subsequent research must justify.

A Feature Request Is a Solution Proposal, Not a Problem Definition
Consider the statement, “We need a new executive reporting dashboard.” It sounds like a requirement, and a delivery team could immediately begin discussing data sources, filters, visualizations, and access permissions. But the statement describes an artifact to build. It does not explain which business decision is difficult, what information is unavailable, who needs it, or why the current workflow fails.
Several different problems could produce the same request. Executives might lack timely information, struggle to interpret existing reports, receive inconsistent definitions from different departments, or need approval from people who are not available when decisions must be made. A dashboard might help with some of these difficulties while having little effect on others.
I would treat the request as useful evidence about stakeholder expectations, not as sufficient evidence about the underlying user problem. The distinction allows the team to respect the request without prematurely accepting its implied diagnosis.
This is especially important in enterprise products, where customers, buyers, administrators, operational users, and executives may have different needs. The person requesting a feature may have legitimate decision authority without being the person who experiences the relevant workflow every day. Product strategy requires understanding both perspectives rather than assuming they are interchangeable.
Problem Framing Is Not an Argument Against Delivery
Problem framing can become ineffective when teams use uncertainty as a reason to postpone every decision. Research has a cost, opportunities have deadlines, and some problems require immediate intervention. If a critical workflow has stopped functioning, a team may need to restore service before it can investigate every possible cause of the broader experience problem.
I would therefore scale the depth of problem framing to the decision’s consequences. A reversible change with low implementation cost may justify a short investigation and immediate measurement. A new platform capability affecting several teams, sensitive information, or long-term product economics deserves greater scrutiny.
The objective is not to replace execution with endless discovery. It is to avoid making large commitments on the basis of weak explanations.
Why Stakeholder Requirements Are Frequently Mistaken for User Problems
Stakeholders are often close to important business information. Sales teams hear objections from prospective customers. Support teams encounter repeated friction. Executives understand strategic commitments. Operations teams know which processes are expensive to maintain. These perspectives can reveal genuine product opportunities, and ignoring them would be a mistake.
The difficulty begins when an observation, interpretation, and preferred solution are combined into a single requirement. By the time the request reaches a product team, the original evidence may no longer be visible. What remains is a feature description accompanied by urgency, seniority, or a revenue expectation.
For example, “Customers keep asking for automated reminders” could refer to several different kinds of evidence. Some customers may have explicitly requested reminders. A sales representative may have inferred the need from repeated objections. Support tickets may show that users miss deadlines. Product analytics may show abandoned tasks, but not why they were abandoned. These signals are related, yet they do not support identical conclusions.
I would not reject the feature request simply because it came from a stakeholder. Instead, I would separate four elements: the original observation, the stakeholder’s interpretation, the proposed solution, and the expected business outcome. This creates a more useful starting point for customer discovery.
How to Translate a Feature Request Into a Researchable Question
A helpful first step is to reconstruct the reasoning behind the request. Ask what event caused the issue to become visible, who encountered the difficulty, what they were attempting to accomplish, and what changed when the problem occurred. Then ask why the proposed feature was considered the most appropriate response.
These questions should not become an interrogation designed to challenge someone’s authority. The objective is to make hidden information available to the product team. A sales leader requesting a reporting feature may have evidence of a recurring commercial objection that the team has never seen. A customer success manager may know that a workflow succeeds only because employees manually correct its failures.
Effective problem framing preserves these observations while separating them from the proposed implementation. That enables the team to investigate whether the request points toward a product opportunity, an operational problem, a communication issue, or a constraint outside the product’s direct control.
The GOV.UK guidance on learning about user needs explicitly distinguishes evidence obtained from people using a service from opinions and suggestions that require further research. It also encourages teams to understand what users are trying to accomplish, how they currently do it, and which frustrations interfere with that goal.
Separating Decision Authority From Evidence Authority
A stakeholder may legitimately decide that the business must support a contractual commitment. That authority does not automatically establish that a proposed interface or workflow is the best way to fulfill the commitment. Similarly, a user can describe a painful experience without necessarily knowing which technical or product solution would address it most effectively.
I find it useful to separate decision rights from evidence quality. Leadership can establish strategic priorities, budgets, and acceptable risks. Product, engineering, UX, and research professionals can investigate the assumptions behind proposed solutions. Customers and users contribute knowledge about their tasks, constraints, experiences, and behavior. Each perspective matters, but each answers a different question.
When these responsibilities are confused, product discovery may become political negotiation disguised as research. When they are distinguished, disagreement can be made more precise: the team can identify whether participants disagree about the business objective, the description of the problem, the strength of the evidence, or the suitability of a proposed solution.
Problem Framing Requires Competing Explanations
One of the risks in product discovery is becoming confident in the first explanation that appears plausible. A team hears a familiar complaint, recognizes a design pattern that could address it, and begins planning implementation. The original hypothesis gradually becomes accepted fact, not because sufficient evidence accumulated, but because the organization started working around it.
Thomas Wedell-Wedellsborg’s Harvard Business Review article on reframing problems examines how changing the definition of a problem can reveal different possibilities. The underlying lesson is not that every apparent problem hides one objectively correct alternative. It is that considering different explanations can reveal options that were invisible within the initial framing.
In my view, competing problem statements should be treated as alternative hypotheses rather than competing slogans. The question is not which statement sounds more compelling in a presentation. It is which interpretation best explains the available evidence, what each interpretation leaves unexplained, and which observations could distinguish among them.
A Hypothetical Example: The Approval Dashboard Request
Imagine a B2B operations platform in which employees prepare purchase requests and managers approve them. A stakeholder reports that approvals are taking too long and requests a new dashboard showing outstanding requests. The suggested product feature seems reasonable: more visibility could help managers recognize work that needs attention.
Before accepting the request, I would consider at least three competing problem explanations. These are hypothetical possibilities, not findings from an actual user study.
| Candidate problem | Possible evidence | What could challenge it? |
|---|---|---|
| A. Managers cannot see pending work | Managers report difficulty locating requests; observation shows repeated navigation and searching. | Managers can readily identify pending requests, yet approvals still remain delayed. |
| B. Responsibility is unclear | Requests move among teams without a consistently understood owner; users repeatedly ask who should act next. | Ownership is already explicit and consistently understood across relevant workflows. |
| C. Approval rules create unnecessary waiting | Requests remain blocked by sequential approvals or conditions that do not match actual decision risk. | Delays mostly occur before requests enter the approval workflow rather than within its rules. |
All three explanations could coexist. The purpose of the comparison is not to force a single cause where several causes may contribute. It is to prevent the team from treating a dashboard as the obvious answer before understanding which mechanism produces the delay.
If the dominant issue is missing visibility, a dashboard or better task presentation might be appropriate. If ownership is unclear, the better intervention could involve role definitions, workflow routing, or communication. If the approval policy itself creates unnecessary waiting, adding visibility may make the delay more noticeable without reducing it.
This is the practical value of problem framing: it changes the discussion from “Which dashboard should we build?” to “Which explanation of the delay is sufficiently supported, and what intervention would address it?”
Look for Positive Exceptions and Counterexamples
When exploring alternative explanations, I would also examine situations where the supposed problem does not occur. Perhaps one department completes approvals without difficulty despite using the same interface. Perhaps experienced users succeed while new users struggle. Perhaps small requests move quickly while more complex requests encounter repeated delays.
These differences may reveal relevant conditions that the initial problem statement overlooked. They can suggest where to investigate workflow rules, knowledge, permissions, data availability, or the relationship between product design and organizational practice.
However, a counterexample does not automatically disprove a problem. A user group can experience substantial difficulty even when another group does not. The significance of an exception depends on whether the groups, tasks, environments, and constraints are meaningfully comparable.
An Evidence Hierarchy for Product Discovery and Problem Validation
Not all evidence collected during problem framing answers the same question. A stakeholder’s account may reveal organizational expectations. A customer interview can explain how someone experiences a difficulty. Product analytics can identify patterns in observable events. A contextual study can expose workarounds that users have stopped noticing or reporting.
I would resist treating these sources as interchangeable or arranging them into a universal hierarchy where one research method is always stronger than another. Evidence quality depends on the claim being investigated. An interview may be well suited to understanding a person’s reasoning but insufficient for estimating how common a problem is across a large population. Analytics may estimate the frequency of an observed event without revealing its cause.
The Nielsen Norman Group’s guidance on research triangulation explains the value of combining methods with different strengths and limitations. For product discovery, triangulation is most useful when the methods investigate related questions while producing distinguishable forms of evidence.
Evaluate the Claim Before Ranking the Evidence
Suppose the team believes that managers miss approvals because they cannot see pending tasks. An interview in which a manager describes searching for requests can support the existence of that experience. Direct observation of repeated searching can support the behavioral interpretation. Event data showing that users repeatedly open several screens before taking action may provide another signal, although its meaning depends on how the events were defined.
None of these observations alone establishes the full causal explanation. Managers might search because requests are poorly organized, because approval ownership is unclear, or because the information required to make a decision is incomplete.
For this reason, I would document evidence according to its origin, relevance, limitations, and ability to distinguish competing problem statements. The goal is not to accumulate more research artifacts. It is to reduce uncertainty about a consequential product decision.
| Evidence source | Useful for | Important limitation |
|---|---|---|
| Stakeholder interviews | Business objectives, operational concerns, contractual obligations, and perceived opportunities. | Secondhand interpretations may not accurately describe user behavior or underlying causes. |
| Customer or user interviews | Understanding experiences, motivations, constraints, and specific past situations. | Recollection and self-report can be incomplete; small samples cannot establish population prevalence. |
| Contextual observation | Actual workflow behavior, workarounds, environmental constraints, and interactions across systems. | Observed participants and contexts may not represent the full target population. |
| Product analytics | Patterns in recorded behavior, event frequency, and changes across defined populations or periods. | Instrumentation may be incomplete; correlation does not establish why behavior occurs. |
| Support and operational records | Recurring incidents, reported friction, exception handling, and visible operational costs. | Only a subset of users report problems, and categorization can introduce bias. |
| Targeted task research | Whether users can complete a defined task and where observable difficulty occurs. | Test conditions can differ from routine use and may not establish long-term business impact. |
Rather than asking which method is generally the most reliable, I would ask which missing evidence could change the decision. If the team already understands why a few users struggle but does not know whether the problem affects a strategically important population, further qualitative interviews alone may not resolve the uncertainty. If analytics show a widespread drop-off but the cause is unknown, more dashboard analysis may not be sufficient.
Investigate Actual Behavior Before Hypothetical Preferences
Customer discovery interviews become more informative when people describe a specific recent situation rather than speculate about features they might use. Ask what they were trying to accomplish, what happened, which alternatives they considered, how they worked around the difficulty, and what consequences followed.
Teresa Torres describes the importance of collecting stories about past behavior and synthesizing customer needs, pain points, and desires in her guidance on interview snapshots. These artifacts can help teams preserve the context of an observed opportunity rather than converting every customer statement directly into a feature idea.
I would still be careful with the interpretation. A concrete story is better evidence for a past experience than a hypothetical preference, but it may not establish prevalence or causal certainty. Interview questions, participant selection, memory, and the researcher’s own expectations can all influence what the study reveals.
What If User Research and Product Analytics Disagree?
Conflicting signals should not automatically be averaged into a single conclusion. A group of interviewed users may describe substantial frustration while aggregate analytics appear stable. That difference could reflect sampling, task frequency, a problem affecting only particular users, incomplete event tracking, or an outcome that analytics were never designed to measure.
Before deciding that one source is wrong, compare the user populations, time windows, task definitions, and observable behaviors. The relevant analytical question is whether both sources are measuring the same phenomenon.
Problem framing becomes stronger when disagreement identifies a new research question. It becomes weaker when the team selects whichever source confirms its preferred solution.
The Problem Framing Evidence Gate: A Decision Framework for Product Teams
I propose the Problem Framing Evidence Gate as a lightweight decision method for determining whether a problem definition is sufficiently credible to guide deeper product discovery or justify further investment. It is my own analytical synthesis of established user research and product discovery principles, not a published standard or a framework validated through controlled research.
The central idea is to make five conditions explicit before a proposed feature becomes the team’s default direction: problem origin, competing explanations, evidence relevance, impact, and disconfirmation. Each condition addresses a different way in which product teams can become confident too early.
Gate 1: Establish the Origin of the Problem
Begin with the original signal. Who raised the issue? What did they observe directly? What interpretation did they add? Which solution did they recommend, and what business consequence are they trying to avoid?
For the hypothetical approval platform, the stakeholder’s request for a dashboard would be recorded separately from the observation that approvals are slow. If no direct evidence exists yet, the underlying problem remains a hypothesis. That does not make the stakeholder’s concern unimportant; it tells the team which part of the claim requires investigation.
I would also identify who experiences the problem and who benefits from solving it. An operational user, a purchasing manager, and a financial approver may have different definitions of success even within the same workflow.
Gate 2: Compare Competing Problem Statements
Next, formulate at least one credible alternative explanation when the decision is consequential or uncertain. The purpose is not to generate arbitrary possibilities. Each alternative should explain the relevant observations and suggest evidence that could differentiate it from the leading interpretation.
In the approval example, missing visibility, unclear ownership, and restrictive approval rules lead to different investigations. The team should not select one because its associated feature is easiest to visualize or has already been estimated by engineering.
Sometimes the competing explanation is that the issue is not sufficiently important to justify product development. A visible inconvenience may be rare, inexpensive to work around, or concentrated in a workflow that is about to be replaced. Problem framing should preserve the possibility that no new feature is required.
Gate 3: Match Evidence to the Claim
For each candidate problem statement, identify the evidence currently available and the uncertainty it leaves unresolved. A user interview may establish that a difficulty exists for one participant. A behavioral dataset may help estimate its frequency among measured users. Workflow observation may reveal a mechanism that neither source alone can explain.
I would label evidence as direct observation, reported experience, recorded operational behavior, stakeholder interpretation, or untested assumption. These are provenance categories rather than numerical confidence scores. Their purpose is to prevent a plausible inference from being presented as though it were an observed fact.
Evidence should also be evaluated for relevance. A frequently reported issue among users outside the intended target market may not support the same product priority as a less frequent but consequential problem affecting the organization’s principal customers.
Gate 4: Establish the Cost of Being Wrong
Not every uncertain problem requires the same amount of research. Before selecting the next action, assess what would happen if the leading problem statement were incorrect. Consider development effort, architectural commitments, opportunity cost, accessibility implications, operational change, and the difficulty of reversing the proposed intervention.
If the proposed change is small, safe, and easily reversible, lightweight investigation followed by measurement may be appropriate. If the proposed solution requires substantial restructuring, affects sensitive workflows, or creates long-term obligations, weak evidence becomes more consequential.
This step does not turn problem framing into a complete product prioritization model. Its narrower purpose is to determine how much uncertainty the organization can reasonably accept before continuing.
Gate 5: Define Disconfirmation and the Next Decision
The final gate asks what evidence would make the team reconsider its leading explanation. If the proposed problem is lack of visibility, what would it mean to discover that users can already find their pending tasks? If unclear ownership is suspected, what would challenge that hypothesis? If approval rules appear to be the cause, which observations would indicate that the delay occurs somewhere else?
Without a disconfirmation condition, product discovery can become a process of collecting supportive examples indefinitely. A team may interpret every interview as confirmation because it never specified what contrary evidence would look like.
I would end the gate with one of four decision states: proceed with a provisionally supported problem, investigate a material uncertainty, reframe the problem, or stop the proposed investment. These are decision categories, not claims that every situation can be resolved with a simple checklist.
| Gate | Core question | Expected output |
|---|---|---|
| Origin | What was observed, and who interpreted it? | Separated observation, assumption, and proposed solution. |
| Alternatives | What other explanations fit the situation? | Competing, researchable problem statements. |
| Evidence | What supports each explanation, and what remains unknown? | Evidence provenance, scope, and limitations. |
| Impact | What would the organization risk by pursuing the wrong problem? | Proportionate research and decision requirements. |
| Disconfirmation | What could change the leading interpretation? | Next investigation and explicit decision state. |
The value of this framework is not in the number of gates. It is in establishing a visible relationship between the original request, the evidence, and the decision. A team should be able to explain why it is exploring a particular user problem, which alternatives remain plausible, and what would cause its direction to change.
How to Write a Problem Statement Without Prescribing the Solution
Once the team has identified a plausible problem, it needs a concise statement that communicates the user, the situation, the difficulty, and its significance. This statement should direct product discovery without prematurely specifying a feature.
The Nielsen Norman Group’s user need statement guidance describes problem statements as a way to establish alignment before solution ideation. It emphasizes the user, the need, and the underlying insight rather than embedding a preferred implementation in the description.
I prefer a problem statement that distinguishes what is known from what remains hypothetical. For a research-stage product, a provisional statement may be more responsible than presenting a confident diagnosis without sufficient evidence.
From a Feature Request to a Researchable Problem Statement
Return to the hypothetical approval platform. The initial request is simple: “Build a dashboard that shows overdue approvals.” That statement already assumes the form of the solution and implies that visibility is the main constraint.
A provisional problem statement could instead read:
Managers responsible for purchase approvals may struggle to identify which requests require their action during multi-step workflows. This could contribute to delayed decisions and repeated coordination between teams. We need to establish whether limited visibility, unclear ownership, or approval rules are responsible for the observed delays before selecting a product intervention.
This statement is not a verified research finding. It explicitly identifies the uncertainty and names competing explanations. That makes it a more appropriate starting point for problem validation than a feature request presented as a settled requirement.
Once research produces credible evidence, the team can refine the statement by replacing speculative language with supported findings. However, it should not claim that the problem affects all customers, produces a specific financial loss, or has a particular causal mechanism unless the evidence warrants those conclusions.
A Reusable Product Problem Statement Structure
For practical use, I would organize a problem statement around the following elements. They are not mandatory fields for every situation, but together they help prevent important assumptions from disappearing:
- Affected user: The specific person or user group experiencing the difficulty.
- Context: The task, environment, or workflow in which the difficulty occurs.
- Observed difficulty: What happens, using language supported by available evidence.
- Consequence: Why the problem matters to users, operations, or the business.
- Uncertainty: What is still unknown about prevalence, mechanism, or impact.
- Discovery boundary: Which question should be investigated before choosing a solution.
The result should remain concise enough for an engineering, design, and product team to use in a real decision. If a statement requires several pages to explain, the team may be mixing the problem definition with its supporting research or attempting to combine multiple unrelated issues.
Equally important, a well-written problem statement should remain open to revision. Its purpose is not to protect the first hypothesis from criticism. It is to give the team a coherent starting point that can improve as better evidence becomes available.
From Problem Framing to the Product Discovery Process
Problem framing and product discovery are closely connected, but they do not perform exactly the same function. Problem framing identifies the challenge worth investigating and clarifies the assumptions surrounding it. The broader product discovery process continues by investigating opportunities, exploring possible responses, testing assumptions, and deciding what to build.
Teresa Torres’s Opportunity Solution Tree framework offers a useful way to connect a desired product outcome with customer opportunities, potential solutions, and assumption tests. The important distinction is that the opportunity space concerns customer needs, pain points, and desires, while the solution space concerns interventions intended to address them.
I see problem framing as a discipline that helps determine whether the team has identified the relevant opportunity before investing in a solution. This does not replace an opportunity solution tree, and the Evidence Gate should not be confused with Torres’s established framework. They serve different purposes: the tree organizes potential paths to an outcome, while the proposed gate examines the credibility of the underlying problem interpretation.
A Valid Product Opportunity Must Connect User and Business Value
A user problem does not become a product priority solely because it exists. The team also needs to consider whether addressing it would advance a meaningful business or product outcome, whether the organization can reach the affected users, and whether the intervention is compatible with the product’s responsibilities.
For instance, a recurring difficulty may be real but caused by a third-party process outside the product team’s control. Another problem may be infrequent but highly consequential for a strategically important customer group. The appropriate decision depends on context rather than the number of complaints alone.
My preferred product strategy is to connect a supported user problem to a business objective without altering the problem description to make an existing roadmap appear justified. User evidence and business constraints should influence the decision together, but they should remain distinguishable.
When Is a Problem Ready for Solution Exploration?
A problem is ready for deeper solution exploration when the team has a sufficiently clear description of the affected users, supporting observations, relevant alternatives, and the uncertainty that remains. This does not require causal certainty or a statistically representative study in every case.
I would also require an explicit reason for the level of confidence being accepted. A low-risk, reversible solution may justify moving forward with incomplete information. A costly platform investment or a change affecting critical user tasks may justify additional research before commitment.
The Atlassian overview of product discovery emphasizes connecting customer evidence, business objectives, product decisions, and delivery rather than treating discovery as an isolated activity. That connection matters because problem definitions should remain open to correction when implementation or real-world use reveals information the earlier investigation missed.
Product discovery should not end when the team starts building. If a proposed solution fails to influence the intended user behavior, the team needs to consider whether the execution was inadequate, the intervention was unsuitable, or the original problem framing was wrong.
How to Test Stakeholder Assumptions Before Building Features
The next step after identifying competing explanations is not necessarily building a prototype. It is selecting the least costly credible investigation capable of changing the product decision. That investigation might involve interviews, observation, existing operational data, a focused analytics review, or a limited technical investigation.
I would start by identifying the assumption that carries the greatest consequence if wrong. In the approval example, the key assumption might be that managers cannot identify pending requests. If observation and existing records show that managers can consistently find pending work, the evidence would weaken the case for a visibility-focused dashboard, even if the dashboard remains popular with stakeholders.
By contrast, if managers repeatedly struggle to locate requests across representative workflows, the visibility hypothesis gains support. The team would still need to evaluate possible interventions rather than assuming that one particular dashboard is the best response.
Choose Research Methods That Can Distinguish Between Explanations
Research should be designed around the uncertainty it needs to resolve. If the question is what happens during a multi-step task, direct observation may be informative. If the question is how frequently delays occur among instrumented workflows, reliable event and operational data may be useful. If the question is why users perceive a process as unpredictable, interviews can help uncover expectations and missing context.
For a meaningful investigation, I would document which user group is being studied, how participants or records are selected, which task or event is being examined, and what limitations might affect interpretation. Research that omits these conditions can produce persuasive anecdotes without sufficient decision value.
Where personal or operationally sensitive data is involved, the investigation must also respect privacy, permissions, and confidentiality. A product team’s desire for better evidence does not justify collecting more identifying information than the research requires.
Define a Research Stopping Rule Without Pretending Certainty
Teams can continue collecting evidence long after additional information is unlikely to change the decision. They can also stop prematurely because the first few findings appear to confirm a preferred narrative. Both create problems.
Instead of imposing a universal interview count or arbitrary confidence percentage, I would define a stopping condition around the decision. What uncertainty must be reduced before the organization can responsibly proceed? What evidence would cause a change in direction? What is the cost of further investigation compared with the cost of making the wrong commitment?
For the hypothetical approval workflow, a stopping rule might require sufficiently credible evidence to distinguish between visibility, ownership, and policy constraints for the target users. It would not require proving that no other contributor to approval delays exists.
Research readiness is therefore a judgment about whether the remaining uncertainty is acceptable for the next decision, not a claim that the problem has been understood perfectly.
Why Technical Feasibility Belongs in the Problem Discussion
Problem framing is often described primarily as a product or UX activity. I think engineering and operational knowledge should also inform it, particularly when the proposed intervention depends on existing infrastructure, data availability, system ownership, or external integrations.
Engineering involvement does not mean using technical constraints to dismiss user needs. It means exposing whether a proposed solution is feasible, what assumptions it makes about the system, and whether the same user outcome could be achieved with less complexity or lower risk.
Consider the approval platform. A dashboard might be technically straightforward if the relevant workflow events and permissions already exist. It could be much more complicated if approval status is distributed across multiple systems with inconsistent ownership definitions. The second condition may reveal that the product team’s original problem framing overlooked a data or process constraint.
Similarly, a feature that appears to solve a user problem may impose ongoing support, maintenance, security, or operational costs. Those implications do not invalidate the user need, but they influence whether the proposed intervention is appropriate.
A Correctly Identified Problem Can Still Be the Wrong Product Investment
Identifying a real problem is necessary for many product decisions, but it is not sufficient to justify every investment. The organization must consider the value of solving it relative to alternative opportunities, the feasibility of a meaningful intervention, and the cost of maintaining the resulting capability.
I would therefore avoid turning user research into an argument that every discovered pain point deserves a feature. Some problems can be addressed through content, communication, service design, operational changes, or the removal of an unnecessary step. Others may fall outside the organization’s current strategy.
Problem framing should leave room for those conclusions. The strongest outcome of a discovery investigation may be a decision not to build the requested feature, supported by evidence that a different intervention or no intervention is more appropriate.
Making Problem Framing Part of Product Decision-Making
A well-framed problem has limited value if it disappears when the team begins delivery planning. The evidence, alternatives, uncertainties, and decision rationale should remain accessible enough for future team members to understand what the organization chose to pursue and why.
I would maintain a concise problem decision record linked to the relevant research and product work. This does not need to become a new software platform or an elaborate approval process. A short document can preserve the original request, the selected problem statement, rejected explanations, supporting evidence, open questions, and conditions for reconsideration.
This is particularly helpful when different stakeholders enter the project at different stages. Without a record of the problem decision, a new executive or engineering lead may interpret the feature backlog as the original business requirement. The team can then end up defending a solution whose underlying reasoning has been lost.
What a Problem Decision Record Should Preserve
For consequential product decisions, I would retain the following information:
- Original request: The feature or change initially proposed by the stakeholder.
- Business objective: The outcome the organization intends to improve.
- Selected problem statement: The user difficulty currently prioritized for investigation or intervention.
- Competing explanations: Other plausible problem definitions and why they were not selected at this stage.
- Evidence and limitations: Relevant research sources, observed behavior, assumptions, and unresolved uncertainty.
- Decision ownership: Who is accountable for the priority and which specialists contributed evidence.
- Reconsideration conditions: New observations or changes that would justify reframing the problem.
This structure can also improve conversations between product managers, founders, designers, and engineers. Instead of reopening the entire discussion whenever someone questions a feature, the team can identify which assumption is being challenged and whether new evidence changes the original decision.
When Should the Problem Statement Change?
A problem statement should change when new evidence materially alters the team’s understanding of the affected users, the mechanism causing the difficulty, or the importance of the outcome. Changing the statement is not necessarily evidence that the original research failed. It may indicate that the team has learned something meaningful.
However, the team should distinguish genuine learning from convenient reframing. If a proposed feature fails to deliver the expected result, rewriting the problem statement solely to declare success would undermine the integrity of the product discovery process.
I would preserve the reasoning behind substantial changes: what was previously believed, what was observed, and why the new interpretation is better supported. This creates a traceable relationship between product learning and the decisions it influences.
Common Problem Framing Mistakes in Product Teams
Starting Discovery With a Preselected Solution
When the first research question is whether users like a proposed feature, the team has already narrowed the problem space. A usability test can reveal whether people can operate a design, but it does not automatically establish that the design addresses the most important problem. Product discovery should preserve opportunities to investigate whether a different response would provide more value.
Treating Stated Preferences as Verified User Needs
A user may genuinely prefer a proposed feature while their actual difficulty remains unresolved by that feature. Preference research has value, but it should not be confused with behavioral evidence or causal explanation. Ask what task the user is trying to complete and what happens without the proposed solution.
Using Research to Confirm Stakeholder Assumptions
Research loses decision value when the questions, participants, or interpretation are selected primarily to support an existing conclusion. I would require teams to identify contrary evidence, alternative explanations, and the limits of their methods before presenting a problem as validated.
Defining a Problem Too Broadly to Investigate
Statements such as “Users need a better experience” or “Customers want more efficiency” may describe desirable outcomes, but they do not identify a sufficiently focused discovery question. A useful problem statement needs a recognizable user, context, difficulty, and consequence. Otherwise, almost any feature can be presented as a response.
Defining the Problem as the Absence of a Feature
“Users do not have a dashboard” describes the current product’s functionality. It does not explain what users cannot accomplish or which difficulty the missing dashboard produces. A stronger problem statement describes the task and obstacle without assuming that adding the requested feature is necessary.
Confusing Organizational Agreement With Evidence
A group of experienced professionals can agree on an explanation and still be wrong. Agreement is important for coordinated action, but it should not be treated as a substitute for relevant customer discovery and user research. I would rather have a clearly documented provisional decision than an apparently unanimous conclusion whose assumptions were never examined.
Evidence and Methodology: What This Framework Can and Cannot Establish
This article draws on established work concerning user needs, UX problem statements, research triangulation, problem reframing, continuous discovery, and the relationship between customer opportunities and product outcomes. The principal sources include the Nielsen Norman Group, Design Council, GOV.UK Service Manual, Teresa Torres’s Product Talk, and Thomas Wedell-Wedellsborg’s work on problem reframing.
The Problem Framing Evidence Gate is my proposed synthesis of those principles. Its five gates, decision states, and application to the approval workflow are analytical tools developed for this article. They should not be interpreted as a formally validated industry methodology or evidence of a completed client research project.
The approval dashboard scenario is entirely hypothetical. No participants were interviewed, no production event logs were analyzed, and no actual product performance or business improvement was measured for that example. The competing problem statements illustrate how an investigation could be structured, not which explanation would be supported by real data.
To evaluate the practical value of the proposed framework, a team could compare how accurately and consistently reviewers identify assumptions, competing explanations, decision-relevant evidence, and unresolved uncertainties when using the Evidence Gate versus their existing process. Such an evaluation would need a defined study design, appropriate participants, comparable decision cases, and explicit criteria for interpreting the results.
Even a positive result from that study would not guarantee superior product outcomes. The quality of the underlying research, organizational decision-making, technical execution, and product-market conditions would remain important influences.
A Practical Problem Framing Checklist Before Product Development
Before a team commits to a consequential feature request, I would use a short review to establish whether the problem is sufficiently understood for the next decision. This checklist should not become an additional approval layer for routine work. Its purpose is to make hidden assumptions visible while the team can still change direction without major cost.
- Separate the request from the problem. Record the stakeholder’s proposed feature and the underlying difficulty as different statements.
- Identify the affected users. Specify whose workflow, behavior, or outcome is relevant, including differences between buyers, administrators, and everyday users.
- Define the business consequence. Explain why the organization should care without inventing a financial or performance impact.
- Write competing explanations. Identify plausible alternatives to the initial diagnosis where uncertainty justifies investigation.
- Inspect existing evidence. Distinguish observation, reported experience, recorded behavior, and interpretation.
- Choose the next research question. Investigate the uncertainty most capable of changing the product decision.
- Evaluate the cost of being wrong. Match research depth to the consequences and reversibility of the potential investment.
- Define a decision state. Proceed, investigate further, reframe, or stop the proposed investment.
- Preserve the rationale. Keep the selected problem statement, evidence, limitations, and reconsideration conditions accessible to the team.
The most useful outcome is not necessarily a perfectly worded problem statement. It is a shared, evidence-aware understanding of what the team is trying to improve, why that outcome matters, and what uncertainty still stands between the organization and a responsible product decision.
Frequently Asked Questions About Problem Framing and Product Discovery
What is problem framing in product management?
Problem framing is the process of identifying, defining, and questioning the problem a product team intends to solve. It separates user difficulties from proposed features, considers alternative explanations, evaluates relevant evidence, and clarifies the scope of product discovery before the team commits to a solution.
What is the difference between problem framing and problem validation?
Problem framing establishes a researchable explanation of the difficulty, including the affected users, context, and competing hypotheses. Problem validation investigates whether the proposed explanation is supported strongly enough for a particular decision. The two activities can overlap and repeat as new information changes the team’s understanding.
How should a product manager respond to a stakeholder feature request?
A product manager should acknowledge the request, identify the business outcome behind it, and ask which observations support the proposed solution. The team can then distinguish the stakeholder’s interpretation from direct evidence about user behavior. The goal is not to dismiss the request, but to evaluate whether it addresses the relevant problem or whether a different intervention would be more appropriate.
How do you write an effective product problem statement?
An effective problem statement identifies the affected user, the situation or task, the observable difficulty, and why that difficulty matters. It should avoid prescribing an implementation and should distinguish verified findings from assumptions. Its scope must be narrow enough to guide meaningful research while remaining open to revision.
Can user interviews alone validate a product problem?
Interviews can provide valuable evidence about individual experiences, motivations, and specific past events, but they do not automatically establish how common a problem is or prove its cause. Depending on the decision, observation, behavioral analytics, operational records, or other research methods may be needed to address remaining uncertainty.
Does every product feature require a formal discovery process?
No. Discovery effort should be proportionate to uncertainty, potential impact, and the cost of reversal. Small, low-risk changes may justify lightweight investigation and measurement. High-impact commitments affecting multiple users, systems, or business processes generally require more explicit evidence and decision reasoning.
How does an opportunity solution tree differ from problem framing?
An opportunity solution tree connects a desired product outcome to customer opportunities, possible solutions, and assumption tests. Problem framing focuses on determining which interpretation of a user difficulty is credible and worth investigating. The activities complement each other, but one is not a substitute for the other.
Conclusion: The Best Product Decision May Begin by Rejecting the First Problem
A product team can spend considerable time improving its technical execution without questioning whether the work itself addresses the right difficulty. Better project management, faster development, and more polished interfaces cannot compensate for a problem definition that was never adequately examined.
For me, problem framing is a discipline of intellectual responsibility. It means treating stakeholder requirements seriously without accepting every proposed solution as a validated need. It means recognizing that different forms of user research answer different questions. And it means allowing evidence to challenge the original diagnosis even when the organization has already become attached to a particular feature.
The objective is not to achieve certainty before product discovery. That would be unrealistic. The objective is to make uncertainty visible, compare credible explanations, and commit resources in proportion to what the team knows and what it risks getting wrong.
Sometimes the evidence will support the original request. Sometimes it will reveal a more important user problem. Occasionally, it will show that the proposed investment should not proceed at all. Each of these can represent a valuable product decision when the reasoning is defensible.
I would rather see a team reconsider an attractive feature before implementation than discover after launch that it built an excellent response to a problem users did not meaningfully have. Good problem framing does not make product development slower by definition. It makes the purpose of development clearer, so that engineering, design, research, and product strategy can work toward an outcome worth pursuing.
For more work on product decisions, discovery, and strategic execution, explore the Product & Strategy Articles collection. You can also find my broader professional perspective on my About page and connect with me on LinkedIn.




