In the previous entry in this series (see link here) we studied if it was possible that the different styles of playing surface might actually be correlated with increased or decreased luck. In this series, we define luck as the number of actual goals in a game that either exceed or fall short of the expected goals metric (which relies on statistical measures of the likelihood of goals given certain measurable activities in a game). We look at luck for both home teams and away teams, because we all intrinsically understand that home teams tend to have better luck when playing at home. See the previous entry to read about what we found.
Today, we’ll do one more evaluation (probably not the last, but definitely one that piques my interest) to determine if individual head referees presence in a game is correlated with greater or lesser luck. The reason we look at head referees only is that I have a data source which lists who the head referee is for a very large set of games. In theory, the head referee controls the flow of the game and contributes the most to uncertainty of the outcome. We’ll look at officiating for both the MLS and the Premier League to see if 1) certain refs affect the luck metric more often and 2) if the impact to luck is significant or not.
MLS 2023
The methodology here in general is to evaluate how both “home luck” and “away luck” can be grouped across the individual head referees. Then we take the mean value of luck and also the standard deviation (how much the luck values tended to vary from game to game that the individual officiated). These are plotted in a similar way to how we plotted the field surface plots. Keep in mind that we’re attempting to describe the entire distribution of games that an individual official was part of. Since we make the assumption that this distribution follows a Gaussian distribution (bell curve) we believe we can describe the impact across all the games with just the mean luck value and the standard distribution. Below we can see the results, where the square describes the mean and the lines describe the variation.
Analysis: I’ll analyze the 2023 MLS results and then leave the analysis of the 2022 MLS games and the Premier League games to the reader. What do I see?
- Remember we’re evaluating how each official impacts home and away luck (remember, luck describes actual goals in excess of the number of goals we statistically predict using the expected goals metric). We see very different mean values of luck for the individual referees, but it is hard to say that the extra “lucky” goals are causal due to the participation of the referee. It takes much more work than this kind of statistical analysis to determine causality. It could be that the referree’s impact is actually correlated with some other event that is more causal about the lucky or unluckiness experienced. That’s just statistician speech to make sure we don’t all grab pitchforks and torches!
- We do see that certain referees are more likely to be associated with higher or lower luck. Some of the referees’ results are close to the average (mean) of the entire distribution of referees. These are mostly the ones in the middle. This means that statistically, the luck experienced during the games are about the same when these “middle” refs officiate.
- However, there are officials off on the edge who do seem to have a statistically higher impact on the luck experienced in the games. The two in red (on the home luck chart) actually are two standard deviations away from the mean value of luck across all the refs. This means that their luck outcomes are different than 95% of all the other referees. This tends to show that the fact that these refs are way out on the edges is not due to chance, but is actually due to something the refs are doing different.
- Note that there are 4 or 5 refs in the Home Luck chart who have a mean impact in their games of close to one goal! You can see that the error bars for these refs vary, but at least one seems to almost always have a one goal impact on a game. One could say that these officials are more likely to give penalty kicks in the box (a very high probability of a score). That might be a good guess, because the expected goals metric that I use actually excludes penalty shots (because they’re random — and therefore “lucky” — events that cannot be predicted). But maybe this metric shows that with certain referees, penalty shots are more probable and are therefore less random.
- Another interesting thing to see is that the luck impact across all the officials is much lower for the away teams. This means that away teams are less likely to be impacted by the presence of individual officials. This is probably reasonable to assert, given the notion that officials in every sport probably have an unconscious bias for the home team (whose fans are screaming about any calls that go against their teams).
- We do see one official whose presence is well-correlated with “good luck” for the home team and “bad luck” for the away team. When I dug in to try to understand why this one official stands out, I discovered that they are very rarely the head official (often getting assigned to do video replays). I also noticed that officials that are outside 67% of the other officials likewise rarely get to be head refs. Perhaps the MLS is paying attention to this (see this webpage for details on this).
Other Charts
MLS 2022
Note the one official that has no error bar? This is most likely because he was the head official only one time in 2022. It’s small data, but observe that it follows the exact opposite trend that we see this official following in 2023! Weird. We also see more dramatic shifts in mean values for the outlier refs in 2022 than we see in 2023.
Premier League 2023
Premier League 2022
Wrapup
So what do you see in the MLS 2022 and the Premier League charts? There are definitely some interesting trends and differences. Feel free to leave comments on what you see and we can dialogue about them!