Opponent-adjusted metrics: Part 2(b), using college football score margin

Introduction

In the previous post, a ranking of all Division 1 college football teams by average opponent-adjusted score margin was presented. However, it was noted that this ranking did not appear to match all that closely to the previously shown ranking of opponent-adjusted record. The reason suggested for that was given as consistency; in fact that may not have been the best way to describe the situation. It is more likely that timing of performances impacts this. For instance, consider Team 1 and Team 2 in the table below.

Each team plays two games against the same two teams. Overall, the adjusted margin of each team is the same, but the adjusted record is not, since Team 1 saved its best performance for a strong opponent.

It is clear that things like this happen: two teams are fairly evenly matched in performance but one team "does just enough" to win a lot of games, whereas the other wins comfortably against weak teams while losing against stronger teams. The reason for this is less clear. It is likely that there is some combination of luck and skill to the distribution of a team's best performances.

In this post, the element of skill in this is ignored and luck is considered as the only driver for this distribution. The 2012 college football season is simulated over 10,000 times to see, according this "luck hypothesis", which teams were lucky to win the number of games they did and which teams were unlucky not to win more.

Method

Following the calculation of average opponent-adjusted score margins in the last post, an opponent-adjusted score margin was calculated for each game played by each team. The average of these for a given team will give the average opponent-adjusted score margin.

For each team, the standard deviation of opponent-adjusted score margins was calculated. It is important to recall that the opponent-adjusted score margin is the margin by which a team might expect to beat an average Division 1 team. Given the average and standard deviation and assuming a normal distribution for opponent-adjusted score margins, a hypothetical opponent-adjusted margin can be generated by use of a random number.

If the hypothetical opponent-adjusted margin for the home team is h, and that for the road team is r, then the performance by the home team is good enough to beat an average Division 1 team by h points and that by the road team is good enough to beat the same Division 1 team by r points. Therefore, if h > r then the home team has won this hypothetical contest; if r > h then the road team has won.

Given the average and standard deviation for any two teams, it is possible to calculate the probability that h > r. This is done for every game in the 2012 college football season (including bowl games). This simulation is repeated more than 10,000 times to find probabilities for the number of games won by each team.

Results

Displaying the probability of winning every number of games (0-13) for all 124 Division 1 teams is not practical. Instead, shown below is the actual number of games won for each as well as the probability of winning more or less than that number. The results are shown below:

Of the 12 win teams (Alabama, Notre Dame, Ohio State and Stanford), only Alabama does not appear to be very fortunate to have won as many games.

In general, the teams which might consider themselves to be unfortunate are those which did not win many games; likewise those which appear fortunate are those which won a large number of games.

Of the teams with a greater than 50% chance of winning more games in the simulation than real life, the one which won the most games was Michigan, which won 8 games, but whose most likely outcome in the simulation was 9 wins (Michigan in fact had a 0.06% chance of winning 13 games!).

Of those teams with a greater than 50% chance of winning fewer games in the simulation than real life, the one which won the fewest games was Wake Forest, who were fortunate to win as many as 4 games - the probability of a winless season for Wake was 2.63%.

Conclusion

While average opponent-adjusted margin, as presented in the previous post, is a useful metric, is becomes much more powerful when combined with the standard deviation, as has been done here. This allows us to calculate whether teams have been fortunate or unfortunate over the course of the season.

The main shortcoming here is in attributing the distribution of better and worse performances over different games solely to luck. While luck must inevitably play a part, it is probable that skill - or, perhaps more accurately, psychology - plays a large part: some teams can do "just enough", others can not.

Of course, the results presented here are entirely retrospective and a statement such as "Wake Forest had a 2.63% chance of a winless season" is close to meaningless: Wake Forest won 4 games, so calculating the probability of a winless season now would give 0%. However, in future seasons, it will be possible to calculate metrics such as these as the season is in progress which will hopefully provide useful forecasts for upcoming games as well as the season as a whole. In this case, it may be appropriate to include a weighting system so that more recent games are more heavily weighted. This would allow a team's progress (or regression) throughout the season to be captured. A method for doing this is something I intend to investigate.