How reliable is your data?

This picture popped up a few times on my LinkedIn feed a few times last week:

It bothered me the first time I saw it but I didn’t give me much attention. The second time I saw it, I looked a little more closely and it became apparent why it had annoyed me…

It’s wrong.

The stats shown are in 2 parts. Both are wrong – let’s look at why:

Part 1:

This is a ‘whole of population’ statistic. It’s talking about all Sales People. 48% don’t follow up at all, 25% make a second contact and stop, 12% make 3 contacts and stop and 10% make more than 3 contacts. So why doesn’t that add up to 100%? It only adds up to 95%. What do the other 5% do?

“And 5% of salespeople spontaneously combust into flames upon securing a job as a salesperson”

No matter what you try and think of, these missing 5% should be counted in the above somewhere. If they have a sales role that doesn’t require them to follow up (really?) then wouldn’t they be captured in the 48% who never follow up? I can’t think of a statement that would cover the missing 5% here – can you?

Part 2:

OK – so this second set of data DOES add up to 100% (hooray!). But look at the numbers more closely. It’s too specific. It says that 80% of sales are made on the 5th to 12th contact but leaves no room for any sales after that. Taken at face value – these stats are saying that no one EVER makes a sale after the 12th contact. I know for a fact that’s not true.

I admire the point the graphic is trying to make, but why do it with rubbish data? Surely there is some real data that supports what they are trying to say?

I googled the “National Sales Executive Association” to find out more about the organisation credited with the graphic – turns out I’m not the only one sceptical. Take a look at these great takedowns here and here.

Another graphic or statistic I see a lot is the ‘systems thinking’ one. Usually attributed to Edwards Deming – the quote is 94% of process problems are caused by the system, not the people. Just look at that cheeky smile.

94%…?

Let’s think about what that means. It means one of two things:

1. That almost all problems have to be 100% the fault of the system so that when we have a problem that is only 60%, or 70% or 80% to do with the system, we can still average out at our lovely 94% statistic. Or:
2. That there is almost no variation in this statistic and process comes in between 90% and 100%, evenly distributed around those figures to average in at 94%

I don’t believe either of these to be likely. It’s certainly not our experience when it comes to deciding whether a business problem is due to the system or the people.

At a guess (and I would only be guessing) I’d say it’s more around 50% either way. I’d be guessing because I simply haven’t evaluated all the business opportunities we’ve ever worked on and classified them as System driven or People driven issues (although that would be a fascinating study!).

Again, I admire the point they are trying to make. If you see an issue with performance, don’t immediately assume that it’s people performing badly – look to the system created and see if you can fix that first. It’s sound advice and doesn’t need a poorly conceived statistic to make it true.

PS – I love Systems Thinking – definitely not hating on it.

PPS – if anyone would like to share a study of a sample of business processes across a range of organisations that proves the 94% statistic, I’d love to read it!