Opponent-adjusted metrics: Part 1, college football record

Introduction

As discussed in my last post, one of the areas I am keen to study is that of opponent-adjusted metrics. Simply knowing a the value of a given metric, in isolation, is not really of use. Opponent-adjustment allows the values of a metric to be compared on a level playing field. I am planning a few posts to discuss a method of opponent-adjustment which I have formulated. In each post, the principals will remain the same, but the detail of the method will be tweaked slightly to accommodate the different types of metric to which it is being applied.

These methods are valid across all sports. The first metric tackled is perhaps the simplest; that of opponent adjusted record in college football.

The Problem

There are 124 teams in the Football Bowl Subdivision. Each team plays 12 regular season games with some conferences following these with a championship game, adding a 13th game for a number of teams. Following this, teams are selected for bowl games, including the National Championship Game, which features those teams ranked 1 and 2 in the country.

Rankings are determined by a combination of computer models and a human polls. However, as each team only plays a small fraction of the total pool, it is difficult to compare teams. An opponent-adjusted system would make this easier.

The general method

In each post in this series, the method will follow the following procedure:

• Calculate the raw metric for each team/player
• Calculate the expected value of this metric, if an average team/player had played the same schedule
• From these two pieces of information, an adjusted metric is calculated which satisfies these crucial criteria:
• If a team matches the expected value of an average team, it is therefore average and the opponent adjusted metric will reflect this
• If a team exceeds the expected value, it is above average
• If a team does not meet the expected value, it is below average

 Specific method

A college football teams record is bounded above by 1.000 and below by 0.000. This is different from, for example, passing yards per game, which is not bounded above; and score margin, which is (theoretically, at least) not bounded above or below.

To accommodate this, the formula for transforming the independent variables, raw value and expected value is chosen accordingly. The considerations are:

• When raw value = expected value, adjusted record = 0.500
• When raw value = 0.000, adjusted record = 0.000
• When raw value = 1.000, adjusted record = 1.000

The final two considerations are difficult, as a team which plays exclusively poor teams but wins every game will have an adjusted record of 1.000. While this is difficult to come to terms with, for a team which does not lose, it is impossible to see with certainty how good the opposition would have to be before they would lose a game. The converse is true for teams which win every game.

The formula chosen for this is of the following form:

• If expected record >= 0.500,

adjusted record = actual record^(log[base (expected record)](0.5))

• If expected record < 0.500,

adjusted record = 1 - (1 - actual record)^(log[base (1-expected record)](0.5)).

Graphs are shown below for adjusted record against actual record, for a number of values of expected record:

 Calculation of the expected record uses, as an initial approximation, 1 minus the average of the raw records of the teams played. From this adjusted records for all teams can be calculated and these values used in the calculation of expected records. This is iterated until stable solutions are reached.

Results

The process outlined above was applied to all games played between FBS teams in 2012-13 (bowl games were included). Games against FCS teams were not included. The results are shown below, with the official BCS rankings shown alongside (these were updated before the bowl games). Note that Ohio State were banned from the postseason so were excluded from the official BCS rankings.

Evaluation and Comments

At a glance, the adjusted-records generated seem reasonable, placing teams near the top that one might expect. The order of the top 25 is different to that of the BCS rankings, but that is to be expected. It is notable that the BCS rankings appear to over-rate Kansas St, Oregon St and Northern Illinois. For each team, this can be explained by a bowl loss (to Oregon, Texas and Florida St respectively), which the adjusted ranking takes into account but which the BCS ranking does not. Likewise, the BCS rankings under-rate Texas A&M, San Jose St and Louisville, who each won bowl games against Oklahoma, Bowling Green and Florida, respectively.

It is important to note that this ranking system is purely retrospective: Adjusted record gives a way to compare performance to date on a level playing field. It is not predictive; it does not hypothesize about which team might beat another on a given day in the future. For example, for much of the season, this method would have ranked Notre Dame above Alabama, because Notre Dame's record, given the teams they played, was superior to Alabama's. However, Alabama was clearly superior when the two sides met in the National Championship Game.

This ranking is still useful, however, as teams can only be judged on achievements throughout the season. Hypotheticals about future matches and conjectured power ranking are just that - conjecture. When choosing teams for bowl games or the National Championship Game, an objective view should be taken, based on performance throughout the season to date and that is something that is achieved by this method.

Having said that, an improvement could be made. The method presented here considers only wins and losses. It is known that scoring margin provides greater discrimination than simple wins and losses when comparing teams, so a method of opponent-adjusting the winning margin would be even better than the method presented here. That is what I will present in my next post.