I’m not a big fan of focus groups or traditional research, except to the extent that it’s usually all we’ve got. The problem with them is that they’re skewed to provide the opinions of people who will give their time away for $40 and stale croissant, and the results can easily be misinterpreted. Beyond that, knowing what we want is not a trait that is particularly strong within humans. The difference between “I like the orange logo” and which logo will have the largest impact on sales can be huge, and extremely difficult to measure.
Patrick Collison, CEO of the online payment platform Stripe, makes this point much more eloquently in a Quora question about his company. On his strategy of observing users in person from the outset and spending time personally overseeing clients integrating their solution, he says:
All of this was useful not merely because it was effective at acquiring users but also because it was an invaluable source of product feedback. After helping someone integrate I’d always head home with a long list of changes I thought we should make. I don’t so much consider this a smart hack as something that it’d have been crazy not to do. Without constantly sitting beside users, we’d have been stuck with things like coarse metrics and the occasional email or comment — all extraordinarily low-bandwidth feedback channels.
Metrics and research are important, but personal interaction with your customers to really understand their needs and how they want to interact with your product? That’s the real data you need to succeed.