Never mind about Digital Strategy or Skills Gaps or Cybercrime or C-level Influence. My favourite statistic in the CIO Survey from Harvey Nash & KPMG is that 77% of CIOs believe they are performing in the top 20% of their peer group.
77 CIOs can’t sit around a top table with only 20 chairs. So what’s going on? The question posed by the survey of 3,352 CIOs was this:
“Thinking objectively about your success in your current role, would you say you are performing within the top 20% of your IT leader peer group?”
The CIOs surveyed over-estimated their performance relative to their peer group by some margin. If 20 of the 77 were actually in the top 20th percentile then 57 got it wrong. They thought that something had a 0.77 chance of occurring when the real chance was 0.20 – a near-reversal of the actual odds. What if this were a bet on the success of an outsourcing deal or a digital transformation initiative?
The confidence interval for this proportion is very tight so this isn’t a random sampling error. If asked this question, most CIOs really would over-estimate their performance.
It would certainly be interesting to see this picture within subgroups such as country, sector, company size and gender. Women are generally less over-confident when estimating the outcome of events. But are American CIOs in financial services more rational than their European counterparts in FMCG?
More importantly, should CEOs be worried that over half of CIOs could make such a seemingly irrational judgement?
No, I don’t think so.
The reason may be evolutionary. We may be seeing an example of the Illusory Superiority bias, also known as the ‘above-average-effect’. The classic example is that about 80% of drivers rate their abilities as ‘above average’ in a range of dimensions.
We tend to rate ourselves more highly than we rate other people, especially groups. We dismiss those painful failings in ourselves which seem so apparent in others. We hear the reasonable voice inside our own heads but we can’t hear the voice playing inside theirs.
It may be that the vagueness of concepts like ‘performance’ and ‘peer group’ in the question also allowed the selection of more salient, positive self-perceptions even when respondents were asked to ‘think objectively’.
I applaud Harvey Nash & KPMG for including this question for what appears to be the first time in the 18 year history of the survey.
This single question reveals why evidence-based decisions are critical in even that most left-brain of business functions. IT leaders are not objectively better at making judgments in the presence of cognitive bias.
Why? Because CIOs are only human.