A consistent weighted ranking scheme with an application to NCAA college football rankings

The National Collegiate Athletic Association (NCAA) college football ranking, in which the so-called national champion is determined, has been plagued by controversies the last few years. The difficulty arises because there is a need to make a complete ranking of teams even though each team has a different schedule of games with a different set of opponents. A similar problem arises whenever one wants to establish a ranking of patents or academic journals, etc. This article develops a simple consistent weighted ranking (CWR) scheme in which the importance of (weights on) every success and failure are endogenously determined by the ranking procedure. This consistency requirement does not uniquely determine the ranking, as the ranking also depends on a set of parameters relevant for each problem. For sports rankings, the parameters reflect the importance of winning vs. losing, the strength of schedule, and the relative importance of home vs. away games. Rather than assign exogenous values to these parameters, we estimate them as part of the ranking procedure. The NCAA college football has a special structure that enables the evaluation of each ranking scheme and hence, the estimation of the parameters. Each season is essentially divided into two parts: the regular season and the postseason bowl games. If a ranking scheme is accurate, it should correctly predict a relatively large number of the bowl game outcomes. We use this structure to estimate the four parameters of our ranking function using "historical" data from the 1999-2003 seasons. Finally, we use the parameters that were estimated and the outcome of the 2004-2006 regular seasons to rank the teams each year for 2004-2006. We then calculate the number of bowl games whose outcomes were correctly predicted following the 2004-2006 season. None of the six ranking schemes used by the Bowl Championship Series (BCS) predicted more bowl games correctly over the 2004-2006 period than our CWR scheme.