A few days ago Alexis Brion shared this article with me: “The limits of the Startup Method“.
One of the main points of the article is that too much testing can be detrimental: “too much feedback from customers might cause the entrepreneurs to change the idea so frequently that they become disheartened”
Along with that, the author suggests that “the lean startup method might be producing “false negatives,” meaning good ideas are mistakenly rejected because the approach does not have a clear rule for when entrepreneurs and intrapreneurs should declare victory, stop testing, and begin scaling production.”
All in all, it is a great article that got me thinking about a few things:
First and foremost I think the key concept behind testing is focus and purpose: you need to understand that you test in order to answer a specific business question and that the result of the testing is direction, not conclusive information. If you lose track of this then yes, testing will lead to more confusion than certainty.
Also, it reminded me of the reasons we test with up to 6–8 people per profile. The ROI (return on information) diminishes greatly after the 6th person. Don Norman and others have written extensively about this.
The author has a point when he says that questioning yourself repeatedly leads to endless pivots. But the point here is that you have to be selective on who you listen to: not all feedback is worth the same.
Also related is the fact that when you interview customers almost invariably, you always get requests for features. But it is the skill and good judgement of the person interpreting the information to distill feature requests from insights.
A few days ago Alexis Brion shared this article with me: “The limits of the Startup Method“.
One of the main points of the article is that too much testing can be detrimental: “too much feedback from customers might cause the entrepreneurs to change the idea so frequently that they become disheartened”
Along with that, the author suggests that “the lean startup method might be producing “false negatives,” meaning good ideas are mistakenly rejected because the approach does not have a clear rule for when entrepreneurs and intrapreneurs should declare victory, stop testing, and begin scaling production.”
All in all, it is a great article that got me thinking about a few things:
First and foremost I think the key concept behind testing is focus and purpose: you need to understand that you test in order to answer a specific business question and that the result of the testing is direction, not conclusive information. If you lose track of this then yes, testing will lead to more confusion than certainty.
Also, it reminded me of the reasons we test with up to 6–8 people per profile. The ROI (return on information) diminishes greatly after the 6th person. Don Norman and others have written extensively about this.
The author has a point when he says that questioning yourself repeatedly leads to endless pivots. But the point here is that you have to be selective on who you listen to: not all feedback is worth the same.
Also related is the fact that when you interview customers almost invariably, you always get requests for features. But it is the skill and good judgement of the person interpreting the information to distill feature requests from insights.
