Through the WSJ Accelerators discussion, Brad Feld identifies a culture of experimentation as core to growth:
A long time ago, I realized that every successful business was a continuous process of small experiments that operated in the context of a long-term vision. When an experiment worked, you did more of it. When it didn’t, you ended it and moved on.
The magnitude of these experiments are dependent on the stage and resources of the company. If you are a three person startup with very little money in the bank, your experiments are tiny ones. As you get bigger and have more success, your experiments can get larger.
While it’s easy to miss the forest for the trees, someone has to tend the trees. Jeff Jordan, now a partner at Andreessen Horowitz, shares his experience with data-driven product development as OpenTable’s CEO:
At OpenTable, one of the most highly leveraged examples of an innovation to optimize our core business was developing a rigorous methodology to pursue and assess potential site improvements… Over time, the data-driven product development methodology at OpenTable matured into a highly disciplined testing regimen. Hundreds of tests have been run in the past few years… lots of singles and doubles supplemented the occasional home run to have a highly material impact on the business. I can’t recommend a rigorous data-driven product development process enough to managers of website businesses—it’s extremely low-hanging fruit in the pursuit of growth.
Similar to Feld, Jordan outlines a maturation process that starts small—a successful test here, a couple failures there—but grows over time into an engine for growth at scale. Underlying both perspectives is a sense of humility which, while not necessarily a precondition to successful experimentation, makes it easier to accept (inevitable) failures.
Jordan cites Ron Kohavi’s description of Amazon’s product development process as central to the ideas behind OpenTable’s approach. In Kohavi’s own words, it comes down to listening to customers:
Our experience indicates that significant learning and return-on-investment (ROI) are seen when development teams listen to their customers, not to the Highest Paid Person’s Opinion (HiPPO). We provide several examples of controlled experiments with surprising results.
Amazon calls this a culture of metrics, complementing one of the most durable long-term visions on the internet (reference Jeff Bezos’ 1997 letter to shareholders). It is easy to get sidetracked by trends like big data, but experiments are less about technology and more about product development cadence.
