Just last week, the latest edition of one of the most well-known studies on the German startup scene, the German Startup Monitor 2016 (DSM), was published. But what do the results really tell us? A commentary.
Studies are a bit like weather forecasts: Without them, we could only guess what's happening. But they don't give us any real certainty. Each one says something different; some are right, some are wrong. And we often don't know which one is ultimately right until it's too late. That's precisely why it's important to look closely at studies if we want to learn something from the results.
Are startups migrating from the centers to the periphery?
This is especially true for large-scale and official-looking studies such as the DSM 2016: The publisher of the publication is the Federal Association of German Startups, Federal Finance Minister Wolfgang Schäuble welcomes the reader, public interest is great.
And right at the beginning of the publication, the first result of the study is quite impressive:
“Overall, it can be seen that the data basis is now more broadly spread across Germany than in previous years.”
In other words, fewer startups surveyed are located in the known hotspots, while more are located in the periphery. For example, Berlin's share of all German startups surveyed in the DSM plummeted from 31.1% to 17.0%. Munich plummeted from 11.5% to 7.0%, and Hamburg from 8.3% to 6.4%. In the Hanseatic city The figures are already the subject of a debate about whether the state government's start-up support has been completely misguided or successful — depending on the party-political perspective.
In contrast, the Rhine-Ruhr metropolitan region rose from 10.3 million to 14.1 million, as did the Stuttgart/Karlsruhe region from 9.9 million to 12.4 million. For the first time this year, the DSM also includes the Hanover/Oldenburg region, where it finds 6.9 million of all German startups—almost the same number as in Munich and more than in Hamburg. Anyone familiar with startup activity in Germany may be surprised, even without detailed knowledge of startup activity in the Hanover/Oldenburg area.
How such distributions came about is a matter of speculation: Perhaps interest in participating in the study has waned in the startup hotspots, or many startups didn't have time to participate. Perhaps the study was promoted with varying degrees of regional intensity. Of course, previous perceptions, including previous editions of the DSM, could also have been inaccurate. However, this would contradict much of the experience of founders and participants in the startup scene, the results of other studies, and common sense.
Unknown population, uncertain sample
Anyone who has ever sat through a statistics seminar knows that a lot depends on the size and quality of the sample in quantitative surveys. Taking Munich as an example, if you calculate the absolute numbers, you can see that 122 startups participated in the study last year and 86 this year. A look at the Munich Startup Map shows: Although the map is far from complete, it already lists 354 Munich startups. This means there are at least four times as many startups in Munich as recorded in the DSM.
The study itself states that because of the unknown population, i.e. the true number of all startups in Germany, the study cannot claim to be representative:
“The DSM 2016 […] therefore primarily serves to first impression of the startup scene in Germany.”
However, this leaves little to be desired from the supposed revelation of a shift in startup activity in Germany from the centers to the periphery. Ultimately, the ranking only reflects the composition of a sample that is, in geographical terms, only partially representative.
Be careful when interpreting
But how can we do it better? How can we achieve a better sample when we simply don't know the total number of startups? This is hardly achievable with a reasonable amount of effort.
Therefore, it is the interpretation of such study results that should be criticized, not the work of the study authors themselves. Although figures presented down to the decimal point may give the impression of absolute accuracy, this could always be due to sampling bias. Incidentally, this does not preclude the fact that the other results of the DSM are extremely exciting and provide a good, in-depth, unique insight into the startup scene in Germany.
And just like with the weather forecast: Better a potentially wrong result than no result at all. However, the weather forecast is no substitute for common sense. After all, we take an umbrella when it rains—even if the forecast calls for bright sunshine.
The German Startup Monitor 2016 is available for free download here.
