Why "would you use this?" doesn't work
By the time you ask, you're a little in love with the idea and want the answer to be yes. People can feel that, and they're polite, so they give you the yes you were fishing for. "Sounds cool, I'd probably use it" costs nothing to say.
Money doesn't protect anyone from this. In 1985 Coca-Cola ran close to two hundred thousand taste tests before New Coke, and the tests said yes. The old formula was back within three months. Quibi raised almost two billion dollars on research saying people wanted short premium video on their phones, and shut down about six months after its 2020 launch. Both asked people what they thought instead of watching what they did.
1. Ask about the past
The one rule I follow in these conversations is to ask about what people have done, not what they would do. A question about the future invites a generous guess. A question about the past asks for facts, and nobody can politely make up a detailed history.
- When did you last run into this? Tell me about that time.
- What did you do about it, step by step?
- What have you tried before, and why did you stop?
Your idea barely comes up, and early on it shouldn't come up at all. The moment you pitch, people start being kind instead of accurate. The conversation guide has the full set of questions on one page.
2. Find five to ten people
You don't need fifty. Patterns show up early, and after seven or eight conversations you'll mostly hear the same things. Look in the communities and forums where people have already described the problem in public, and send a short, plain message. You're researching how people handle it, you're not selling anything, and you'd like twenty minutes of their experience.
Be careful with friends. They're rooting for you, which makes them the least reliable people to ask. And if nobody you contact will talk to you, that's an answer too. If you can't reach the people who have the problem for a conversation, you won't reach them for a sale.
I'm an introvert and would much rather write prompts than ask a stranger for twenty minutes, so this is the step I have to push myself through. It's also the step that decides the most.
3. Score each conversation
I score every conversation on the same scale, weakest to strongest:
- Polite interest. "Sounds useful." Worth nothing.
- A real story. A specific recent time it happened, told with detail. The problem exists.
- Active coping. Workarounds, tools bought, hours spent. The problem is worth effort to them.
- Existing spend. Money already goes to dealing with it.
- A pull. They ask to hear when it exists, offer to try it, or offer to pay, without being asked.
Look for a problem where most conversations land at three or higher. If they cluster at one and two, let that idea go now. It's much cheaper to learn it here than from a launch.
4. Test with a one-page site
Conversations show that the problem is real. A second test shows whether people act when they see an offer. Make one page with a plain description of the solution, in the words people used in your conversations, and one thing to do on it: leave an email, join a waitlist or preorder. Share it where you found your people, or put a small ad budget behind it. What counts is the share of visitors who raise their hand, not how many visitors there are.
If the conversations were strong and the page is quiet, the problem may be real while the way you described the solution is off. That's fixable, and good to learn before you build.
For anything that's a service underneath, there's an older test. Do the job by hand for the first few customers. Zappos began with its founder photographing shoes in local stores and buying a pair at full price whenever an order came in. He lost money on each sale and learned that people would buy shoes online.
5. Decide what counts as a yes before you look
I set the thresholds before any results come in, because afterwards I'll negotiate with myself. Something like this: at least half the conversations reach three or higher, at least one reaches five, and more than a handful of the people who see the page sign up. Change the numbers to fit your idea, but write them down first.
Where AI helps, and where it doesn't
AI is good at everything around the conversations: finding where your people gather, drafting outreach and questions, transcribing calls, and pulling recurring pains and exact phrases out of your notes. Sorting eight conversations used to take me days and now takes about an hour.
It can't stand in for the people. Simulated users and generated "customer feedback" will happily validate anything, because nobody behind the words has the problem. An AI asking the questions is fine, as long as a real person with the problem is answering.
How long it takes
With real attention next to work and life, about two weeks. The first week goes to choosing a shortlist of problems. The second goes to the conversations, the scores and the one-page site. Around a demanding job it takes longer, and fully focused it takes less. What stops it getting shorter is other people. Interviews need scheduling and a test page needs a few days in front of real visitors, and no tool speeds that up.
Related
- The conversation guide, the questions and the one-to-five scale on one page
- The five-question filter, for picking which problem to validate first
Based on chapter 4, "Evidence", and appendix A, "The six-week plan" of The Code Was Never the Hard Part.
