Testbirds tests its customers' software with the help of its own crowd. Since our last visit Some time has passed — and so we asked Philipp Benkler, founder and managing director of Testbirds, to bring us up to date.
Would you still call yourselves a startup?
Of course, what counts as a startup is a question of definition. We now employ almost 100 people. By many common definitions, we are no longer a startup.
But of course it's not just about the size, but also the flair, the way people interact with each other and do things differently than the big guys.
When a company grows, for example, many people shout: “You need hierarchies, you need middle management, you need to bring senior people on board.” We did things differently and continue to rely on flat hierarchies.
“There is no one perfect solution”
Was there a point where informal startup communication on the fly reached its limits?
Communication always becomes a problem as you grow. There were three of us, then four, five, six people in one room. Everyone knew everything at all times. We answered questions across the table.
The first problem arose when we opened a second room next door. Suddenly, no one knew what their colleagues were doing. We introduced a weekly breakfast meeting where everyone talked about what was going on and where they needed help. Very informal, very unstructured. It worked wonderfully.
Until we were 30 people. Everyone sat in a circle and only one person spoke – it was tedious, and people hated it. At first, it was the best communication solution we'd found up to that point, and just a year later, it was complete nonsense. We had to come up with something new.
Today we have multiple offices, are international and communicate in different ways: in person, but also using tools such as Skype, Yammer and Confluence.
What I mean by that is: There is no one perfect solution, but only suitable solutions for certain phases.
©Testbirds/Philipp Guelland
What did you do with the money from your last financing round at the end of 2015?
We bought expensive cars and have lived a good life ever since. Joking aside, for one thing, money went into internationalization. We opened new offices in London, Amsterdam, and Stockholm and expanded existing ones. We also invested in technology. This includes our Device Cloud. Through our platform, our customers have access to every smartphone and tablet in the world. This wasn't technologically trivial and cost money.
What about the franchisees in Russia and Hungary?
Our partners are licensed to exclusively market, execute, and deliver Testbirds services in their region. These companies are responsible for local sales, marketing, project execution, and crowd support.
Can't you depict this yourself on site?
The companies approached us, and we saw that they were a good fit. Since we ourselves have limited resources, we prefer to license good partners and get a piece of the pie.
Do you want to continue on the path to becoming a licensor or open up new markets yourself?
We will do both, depending on the attractiveness and potential of a market. However, franchisees must come to us of their own accord. It's difficult to actively recruit in this area.
Are you investing more in cloud applications because crowd testing is not sustainable in the long run?
Quite the opposite: Crowdfunding is working great, it's our core business model, and the core of our DNA. We're incredibly good at it.
But we received a customer inquiry from a payment solution provider back in 2012 or 2013. His technology was causing problems, but only under certain combinations of operating system, browser, and Flash versions. In total, there were just over 1,000 configurations, and he had no idea where the problems existed and where they didn't. Our crowd couldn't reproduce the problem, however, because most users' browsers and Flash versions automatically update to the latest version. We then hacked together a very simple virtual machine builder and made it available to our crowd so they could test the thousand configurations.
The product initially fell into disuse. However, after the market evolved further toward test automation, we integrated it into our platform. Apart from us, there's no one else who can combine these two solutions into one platform.
"When we started talking about the crowd, most people just understood 'coleslaw'"
The initial hype surrounding the crowd was followed by disillusionment: Whether it's a shrinking author base on Wikipedia or failed crowdfunding projects, are there fewer, but more specialized, applications for the crowd?
When we started talking about the crowd, most people just understood "coleslaw." We had to do a lot of educational work. Now the hype is actually just over.
What you're mentioning are fundamentally different topics that can't be compared. We're in the crowdworking space and use swarm intelligence. We have a wide variety of use cases, and it works brilliantly. Previously, one person had to test an application on ten different devices. That was a tedious and boring task. If you let ten people test on one device each, and they have fun doing it thanks to gamification, everyone benefits in the end.
Will self-learning algorithms — keyword machine learning and artificial intelligence — one day replace crowd testing?
This will definitely happen in some ways. As technology advances, algorithms will take over tests that we currently do manually. To be honest, this applies to many types of work, for example, in Industry 4.0 or the construction industry.
For us, this means that simple test cases will certainly soon be executed with a higher level of automation. I don't see more complex test cases being automated in the medium term. And by complex, I mean everything in the IoT area, for example. One problem, for example, is how I can realistically virtualize a connected car service without having to set up a car.
In the area of user experience and user feedback, we are still a long way from being able to replace humans. The questions are: How do I like an application? Is the flow good? What's the experience like? Am I happy? Am I annoyed by an app? I'm still very skeptical about algorithms with artificial emotions. For example, we're working on smart algorithms in a research project with LMU Munich.
Thank you for the interview!
