How a failure to update priors leads to an underestimation of future batting performance
One of the highlights of my Ashes summer was the daily ZaltzQuiz. In the age of Bazball, nextGen thinking, and accessibility – it was good to see a nauseating statistics quiz take centre stage at the top of the BBC live feed each day.
Day Two of the Old Trafford test would be remembered by most English cricket fans as the day England racked up 374 runs in 72 overs (including a destructive run-a-ball 189 by Zak Crawley). But for me, it was the question and the responses to the Daily ZaltzQuiz that linger in the mind.
The answer was 52.3. The average score in all innings over 40 in men's Tests in the last 25 years is 92.3.
The mean response from the online cricket fraternity (at least the 23 selected to be published) was 24.59, and the median 25 (notice how close these values are – more on that later). Both substantially lower than actual answer.
No wisdom-of-crowds effect present amongst this group of cricket nerds – there appears to be a systemic bias to underestimate. When popular opinion and fact differ by a factor of 2 - this gets a behavioural scientist thinking...why?
Averages are mean
The first source of bias lies in how skewed distributions impact mean averages. In fact, the question never explicitly asks for a mean average – but it is implied by the set up.
Averages are used to describe data – and although the mean almost invariably gets the limelight – there are three ways to calculate an average: mean, median, and mode. How representative this summary statistic is of the underlying data will depend on the distribution.
Presented is a density plot of runs added over 40 by England and Australian batsmen since the year 2000. As you can see – the majority of scores post-40 are indeed substantially lower than the mean average. In fact, the median average of these data is 27 – just two runs away from the average public guess. However, the mean average is dragged upwards by large, but less frequent, daddy-tons (in our data – this raises the mean average to 41).
That proximity of the mean guess and median average suggests that people don’t fully consider the impact of very large, but less common innings – something perhaps appropriately coined the ‘Brian Lara Effect’.
This wasn’t unnoticed by some punters…
The median average, and the guesses sent in, are probably more representative of how batsmen fair post-40 and informative for predicting how a player may progress. But the unquestioned sovereignty of the mean makes a mockery of the layman again.
Anchor and Adjust
The other source of bias comes from a canonical cognitive heuristic: anchoring and adjustment. An estimation strategy whereby we start off with an initial idea (the anchor) and adjust our belief based on this reference point – anchor-and adjustment can prevent us from updating our predictions as much as we should.*
A reasonable strategy uses batting average as an anchor. The general rule-of-thumb for evaluating top-tier test batsman is if they averaged over 50. Even amongst some of the greatest players to play the game – this doesn’t rise above 60. This serves as our anchor.
Fans adjust away from the anchor, or in Bayesian vernacular - update their prior, factoring in the batsman already has 40. But if even Steve Smith only averages 60 – then this can’t go that much further than say 20, maybe 25.
It’s in this reasoning that the cricket fan reveals themselves as a naïve Bayesian – failing to fully take into account how a batsman’s average will change given the amount of runs he has already scored.
Our scatterplot shows how the averages of four recent Ashes greats change depending on their current score. Notice how stable their average runs more remains – if they average, say, 45 – then being on 45 doesn’t increase the likelihood of them getting out soon. In fact, the average for all these batsman rises quicker than runs scored, most notably with Alastair Cook – for every one run scored, his average went up 1.11.**
Fans fail to adjust adequately from their anchor, or prior, to account for how an average will rise with current score – leading to the underestimation of future performance.
Bonus Insight - nervous nineties or slack centurions?
Something which seemed to impact fan estimations was the belief that lapses in concentration affect batsman when they pass 50 (and similarly when they hit a hundred). Is this true - or a case of cricket apophenia?
Our data strongly supports the hypothesis that batsman falter after passing significant milestones. Exploring the conditional probabilities of getting out having reached difference scores, players are 8% more likely to be dismissed between 50-54 than between 45-49, and 45% more likely to be dismissed between 100-104 than 95-99.
This contradicts established cricket wisdom that batsman are particularly vulnerable when approaching a milestone (the nervous nineties). If there is indeed an increase in nervous energy, then it only serves to sharpen focus, the blunting of which having passed said milestone leaves batsman overly susceptible to being dismissed.
*In fact, the anchor doesn’t even need to have any relevance: anchor-and-adjust research has evidenced how wine valuations can be impacted by the final two digits of your phone number.
**Indeed, how an average changes depending on runs scored is possibly a better category heuristic than simply mean batting average, in general (but that’s for another blog).
Comments