How to Test an MVP With Real Users Before You Have Real Users

What you need to know

• User testing before launch is not a beta. It is structured observation of a specific type of user attempting a specific task, used to find the gaps in the core workflow before you expose them to a wider audience.
• The most useful testing is unmoderated: give the user the task and watch what they do without explaining or helping. The explanations you feel compelled to give are the product problems you need to fix.
• Five to eight sessions on the same core workflow surface enough recurring patterns to make meaningful changes. You do not need hundreds of sessions at this stage.
• The goal of pre-launch testing is not to validate that the product is ready. It is to find the specific places where it fails, so you can fix the ones that will block activation before launch.
• Users who match your actual customer profile produce more useful feedback than users who are convenient. The effort of finding the right people pays off in the quality of the observations.

The core argument

The typical approach to pre-launch testing is to share the product with friends and colleagues and ask what they think. This produces positive feedback from people who are trying to support you and specific feedback about features they would want to see. It does not produce observations about what a real customer does when they encounter your product for the first time with no context. Those two types of feedback are so different that they require completely different testing approaches.

The approach that produces actionable product improvements is a moderated task observation. You recruit someone who matches your customer profile, you give them a single task that represents the core workflow, and you watch them complete it without providing any help or context. The moment they hesitate, you note where. The moment they do something unexpected, you note what. The moment they express frustration or confusion out loud, you note the words they use. These observations tell you exactly where the product fails to communicate its own value, where the navigation does not match the user's mental model, and where the core workflow requires more explanation than it should.

I ran this process on the first version of Velmora before any marketing or outreach. Five sessions with people who matched the target customer profile, each given the same core task. Three of the five sessions produced the same confusion at the same step in the workflow. That was enough evidence to change the design before launch. The change cost two days of engineering time. Discovering the same problem after launch would have cost two months of activation failure and a redesign that tried to interpret aggregated metrics instead of direct observation.

Common mistakes

1. Testing with people who already know your product. A founder's friends and colleagues bring context that real users will not have. Their feedback reflects that context. Find strangers who match the customer profile.

1. Explaining the product before the test starts. Every explanation you give before the test masks a gap in the product's ability to explain itself. Start the session with only a task description. No product overview.

1. Asking what users think instead of watching what they do. "What do you think?" produces opinions. "Try to accomplish X" produces behavior. Behavior is more predictive of what will happen in production.

1. Running too few sessions to see patterns. One user's confusion could be that user. Three users' confusion at the same point is a product problem. Run enough sessions to see the patterns.

1. Not documenting observations during the session. Memory is unreliable and biased toward the most recent or most dramatic moments. Take notes or record the session during the observation, not after.

Where to start

1. Identify five people who match your target customer profile. Not friends or colleagues, but people who actually have the problem your product solves. Use your network to find introductions if necessary.

1. Write one task that represents the core workflow. "Please try to [accomplish the primary goal] as you normally would." No product context before they start.

1. Run five sessions and document every hesitation, unexpected click, and verbal confusion you observe. The patterns across five sessions are your priority list for pre-launch fixes.

Related reading

• Validate Your App Idea Before Spending Fifty Thousand on Development
• How to Set Realistic User Targets for Your MVP
• How to Decide Between a Beta, an Alpha, and a Soft Launch
• Pre-Build Customer Discovery: 10 Questions That Save Months of Work