Acceptable Patterns
Essays on good thinking and good products.
Lesson 7 of 12: Audience

Audience Selection

Most research fails before the first question is asked, because the interviewer talked to convenient people, not the right people.

The Convenience Sampling Trap

These are all wrong reasons to select participants:

  • They're in an existing research panel (assembled for a different project)
  • They're your most active power users (unrepresentative of the broader population)
  • They left a review or contacted support (overrepresent users with strong opinions or specific problems)
  • Customer success "found us some users to talk to" (no defined criteria)

Primary and Secondary Attributes

Define audience criteria explicitly in two tiers:

Primary attributes: characteristics that make a user relevant to the decision:

  • Tenure (e.g., signed up in last 30 days)
  • Feature usage recency (e.g., used reporting feature in last week)
  • Role (e.g., hiring managers who initiated onboarding)
  • Specific behavior (e.g., users who abandoned at step 3 of checkout)

Secondary attributes: characteristics that add diversity and surface variation:

  • Geography, company size, platform (mobile vs. desktop), team structure, payment tier

Diversity on secondary attributes often reveals critical nuances that would otherwise be missed.

Airbnb host experience: convenience vs. diversity

1
The easy path
Interview active domestic hosts who are responsive to outreach. They're available, articulate, and have opinions.

Don't Filter Out Dissatisfaction

Teams unconsciously prefer "happy users". They're easier to talk to and less emotionally draining. But users who churned, filed support tickets, or abandoned a feature often hold the most valuable insights.

If your research is about improving retention, interviewing only current active users tells you nothing about why people left.

Your product decision is about reducing churn among mid-tier customers. Which sampling strategy is correct?