Data Driven by Design

Participating at a number of boards of startup and growth stage companies provides me with the advantage of seeing how many founders/management think about, present, and use data. By data, I mean usage, cohort, average revenue per customer (ARPU), default rates, lifetime (LTV), virality etc. i.e. data that drives the business.

I find that management typically falls into one or more of the following use cases with respect to their data :

A. those who provide tons of data but don’t use it themselves (generally for a lack of understanding the data or how to make use of it).

B. those that present a bunch of data to meet some obscure need of the Board to see data. 3. teams that use KPI’s (Key Performance Indicators) to drive their business and data supports decision making.

By way of past experiences I’ve had:

1. A mobile apps company would present tons of information, cohorts, LTV, ARPU, ARPPU, DAU, MAU, to the Board every month as part of their reporting pack. The Board was bamboozled by 30+ slides each month. But management could never put together a coherent story of what was actually happening in their company. There was a real failure to see what was really happening in the market, growth rates for the industry, customer acquisition opportunities, clear return on marketing spend and so on. The feedback was always ‘tell me what you want’ or ‘here’s a whole bunch of stuff’ i.e. what would make the Board leave them in peace — ultimately they did not rely on an analytical approach to drive a business that was all about the data. I can say with certainty that, had they used data and KPI’s properly, their company would have been sold at even a greater multiple.

2/ One of our portfolio companies is involved in online financing of small businesses. It drives the entire business off six or so KPI’s. The Board understands the KPI’s and what drives the numbers, so discussions are much more focused as a result. Based on the known behavior and relationships of the KPI’s to the data, the Board can set the risk appetite for lending, marketing, and future capital expenditure. A typical Board pack consists of a single slide Dashboard of KPI’s (with a few supporting slides for drill down, history and forecast trends). KPI’s include a factor rate, collections to sales ratio, delinquencies to loan book and sales ratios, provision for bad debt to profit ratio and loan book to total addressable market ratios. The company drives its business each day off the same dashboard. Whilst a lot of data is used to drive the business operationally, the Board generally has no need to see so much of the detail. As a result, the company is performing ahead of every expectation.

3/ A third portfolio company simply ‘had no time’ to get the data they needed out of their systems. They kept going in a direction that became more worrisome over time, delving into all sorts of theories why they were not being successful. Eventually we had to shout ‘STOP’, and spend two months articulating KPI’s per area of the business. We started to look at the data such as number of transactions, types, merchant behavior, customer patterns, average transaction values and so forth, whereupon we figured out that, in fact, perception and ego were driving the business and not data. Once the data was understood, the real challenges of customer retention, product UX came out, together with the fact that marketing was not targeted correctly and thus customer acquisition cost was completely off. The company lost valuable time and opportunity in changing to a data driven culture, coupled with new management.

So some thoughts:

1/ Data is useless without an understanding of what its telling you and what KPI’s it drives in the business. Figure out the key drivers, then let various data elements determine how each KPI is measured.

2/ Do it for You — you need to convert to the religion of data — data is true (assuming you collate the right data). The tools available from BI software, to MixPanel, AppAnnie, Google Analytics, Tableau and BigBoard are so easy and cheap to implement and use, that its criminal not to use them.

3/ Don’t over-complicate it. You are building a business, not a data analysis foundation! The other day I had a company get so complicated in what they wanted to achieve, that they ended up showing me a ‘cohort table in Bitcoins’ as this was how they ‘normalized results’ to get what they wanted… I pointed out to them that by just changing the KPI represented to the left of each cohort row, our brains would see the causal relationship. Yet I think they spent a month trying to create a table that ultimately did not show anything in real terms or that one could ‘connect’ with.

4/ Building data insight is an iterative process. Start with simple things and then, over time, build additional data elements if they help support the KPI’s you feel are appropriate. Extend KPI’s into each department and those will in turn drill into additional data.

No data, No Money — investors today are even more focused on the data that drives the business. Not because they want heaps of data — as I explained, the investors/board are interested in high level KPI’s, but they want to understand that the founders can collect the right data, make the appropriate decisions from the data, and that the data is available readily so that they, and company employees, can drill down at a moments notice to determine whats going on. I’ve probably passed on 25% of investments this year purely as they could not articulate what the drivers were in their business, or how they would measure success in an analytical fashion.