INSUBCONTINENT EXCLUSIVE:
on helping people express their most creative ideas
Previously, Koen founded Sofa, an Apple Design Award-winning agency that was eventually acquired by Facebook
It is a commonly held belief that startups can measure their way to success
take Framer from seed round to Series B
In my opinion, the bad way unfortunately is often preached on saas blogs, a/b test tool marketing pages, and especially growth hacker
Silver bullets, if you will.The good way is comparable to first principles thinking
Below the surface of your day to day results, your startup can be described by a set of numbers
It takes some work to discover these numbers, but once you have them you can use them to make predictions and spot underlying trends
If everyone in your company knows these numbers by heart, they will inevitably make better decisions.But most importantly, using data the
everything, so most startups start out that way
But when you measure everything, you learn nothing
Just the sheer noise makes it hard to discover anything useful and it can be demotivating to look at piles of numbers in general.My advice
is to carefully plan what you want to measure upfront, then implement and conclude
Later in this article, I provide a clear set of ways to plan what you measure.A/B tests are anti-startupTo make decisions based on data you
Without volume, the data itself is not statistically significant and is basically just noise
To detect a 3% difference with 95% confidence you would need a sample size of 12,000 visitors, signups, or sales
That sample size is generally too high for most early-stage startups and forces your product development into long cycles.While on the
To get reliable measurements, you should only be changing one variable at a time
During the early stages of Framer, we changed our homepage in the middle of a checkout A/B test, which skewed our results
But as a startup, it was the right decision to adjust the way we marketed our product
In general, constant improvements should trump tests that block quick reactionary changes.Understand your calculations
