Random musings from this year's BCS (post Rose Bowl, at least). I have been surprised to see ABC distance itself from the BCS this year. From the BCS selection show onward, ABC commentators have been nothing but critical of USC's omission from the Sugar Bowl. The criticism is fair given how incomprehensible it seems to most fans that the consensus #1 team is not given a chance to play for the championship, and initially was a surprising show of integrity by ABC, which of course is contractually part of the BCS as well. However, the ABC analysts have been suggesting that coaches should ignore their agreement with the BCS and put USC #1 in their final poll. While I understand the frustration with a system that clearly didn't work as desired, the coaches -- of all people -- should know that you can't change the rules in the middle of the game. When they made their agreement with the BCS, they knew that there was a chance of this happening, but agreed to give the BCS champion their #1 vote. Here's to hoping that everyone does what they should. The writers should follow tradition and keep their #1 team ranked #1 after a bowl win (especially a dominating win over a great Michigan team) and ignore the hype of the Sugar Bowl as the championship game. The coaches should follow through on what they promised to do.
Several pro-USC analysis have been calling for the reintroduction of margin of victory into the BCS computer rankings. This is wrong for two reasons. First, the best computer rankings that consider margin of victory rated LSU and Oklahoma #1 and #2, so it should have made no difference in the BCS standings. Second, a football team's goal is to win, not to win by a prescribed score difference. Voters have the luxury of watching games to see how good each team looked. Computers attempting to evaluate games generally do so only by the final score. Margin-of-victory computers make no distinction between a 28-14 win that was evenly played but the victor had a couple breaks, or a 28-14 win in which the victor dominated and was 28-7 until a meaningless late score. Granted that one team's degree of domination of another is highly correlated with final score, but it is not equal to final score, and there are plenty of cases in which the final score is misleading and would cause an incorrect ranking. In the absence of an accurate way to measure dominance without being swayed by irrelevant plays (scores in garbage time, for example), the BCS is best off using only wins and losses in its computer rankings. Furthermore, the computer rankings that use margin of victory put much greater weight on scores than do voters, so using one of those rankings would create a BCS computer ranking systematically different from the consensus polls (even moreso than is currently the case).
Clearly I spoke too soon (see the Nov. 28 notes below). The big debate during the season may have been non-BCS conferences, but the big debate during the bowl season and throughout the spring will be (a) how can the consensus #1 team not be allowed to play for the championship, and (b) how can the BCS be fixed so this doesn't happen in the future.
There are several issues to examine here. First and foremost is the fact that over half of the BCS formula is made up of computer ratings of one form or another. The loss, schedule strength, and quality win factors effectively comprise a computer ranking in and of themselves, and put together, that ranking is about 1/7 as important as either the poll or computer avearge. In other words, the BCS is not equal parts computer and human, but rather approximately 8 parts computer and 7 parts human. Thus it should surprise nobody that in all three cases in which men and machines have disagreed, the BCS title game was determined by the computers.
To set the BCS to its "as advertised" blend of 50% polls and 50% computer rankings, the simplest solutions are to either remove the loss+schedule+QW components entirely, or to use them as an eighth computer ranking. The down side of this is that it is easier to get two polls to agree than it is to get seven or eight computer rakings to agree; this would give more weight to the polls. (In addition, it would increase the likelihood of ties in the standings.)
All of this assumes that it's a bad thing that the computers have the controlling share of the BCS formula, of course. Could it be that the computers are right? (In the interest of full disclosure, my win-loss and my margin of victory rankings put Oklahoma first and LSU second.) The answer is yes, to a degree. Computer rankings are fundamentally designed to answer the statistical problem of what team is best, as judged from its performance. For that particular criterion, obviously a well-designed statistical ranking will be superior to the averge voter.
However, that is only the first consideration made by a person. People prefer to rank teams ahead of teams they beat head-to-head, since that gives an element of "settling it on the field" to the poll. People prefer conference champions from major conferences, as the round robin played in each conference is designed to take care of ranking teams within the conference. People prefer teams that finish strong, a factor explicitly included in selection processes of other sports but omitted from the BCS. Factors such as these -- especially the head-to-head factor -- are very difficult for a computer ranking to handle, as you would only want some factors to apply in certain team-by-team comparisons. In other words, the fact that Colorado beat Nebraska in 2001 and went on to win the Big XII championship is why Colorado was ranked ahead of a statistically-superior Nebraska team in both polls; however the head-to-head bonus used to move Colorado up (or Nebraska down) did not cause Colorado to move ahead of Oregon in the polls.
In summary, it is clear that a significant portion of the discrepancy between polls and computers is due to the fact that poll voters make more sophisticated decisions than do the computers; the statistical superiority of one team over another is merely one of many factors considered by a typical voter. As such, it seems fair to give a higher amount of weight in the BCS to the voters until such time as the computer rankings are able to make similarly sophisticated decisions.
It is worth repeating the point that the computer rankings did exactly what they were supposed to do. From a purely statistical standpoint, USC really was the #3 team prior to bowl games, and the computer rankings accurately reflected that fact. The problem with the BCS was in its design; over half of the weight of the BCS was given to calculations that ignore things humans find important. It is also worth mentioning that, considering wins/losses, opponents, and game locations, there was a 40% probability that USC was better than LSU or Oklahoma. While the computers correctly recognized USC as the statistically inferior team, they failed to recognize that the degree of inferiority wasn't all that large.
The big debate this season has centered around the plight of teams outside BCS conferences. The charge has been led by Tulane president Scott Cowen, who in July created an organization called Presidential Coalition for Athletic Reform, whose primary goal is to force the BCS to drastically change its system. His goal sounds noble: "We simply want access like we have in all other sports. We want a level playing field. There's not a level playing field in college football. We're not looking for some handout. We're looking for access."
The main arguments used by BCS opponents are patently ridiculous. One is that the BCS formula is stacked against them and unfairly keeps deserving teams out of BCS bowl games. The most frequently-mentioned case is that of Tulane, which went unbeaten in 1998 but did not get a BCS bowl bid. However, Tulane was ranked #10 in both polls and #10 in the BCS rankings; leaving them out of the eight-team affair was hardly proof of unfair treatment. An examination of the top teams from non-BCS conferences over the past eight seasons shows that the BCS rankings do not systematically overrate or underrate those teams, compared with the polls.
There are also those who complain that the granting of automatic bids to some, but not all, conference champions is not only unfair, but unique to I-A football. Those who make this claim are sufficiently ignorant about NCAA tournaments that they do not have the credibility to be arguing about this in the first place. One needs to look only at division I-AA football, the frequently-mentioned alternative to bowl games, where only eight conferences have automatic bids to the tournament. Likewise, the Division I baseball tournament did not give automatic bids to all conference champions until it was expanded to 64 teams. In all such cases, the awarding of automatic bids is done by conference quality, and in the case of I-A football there is a clear-cut difference between the six BCS conferences and the rest of the division - both in terms of top-to-bottom strength (which is what my conference rankings measure) and the ability to regularly produce high-ranked champions.
Finally, it's unclear what system the Tulane president would prefer to the BCS. I don't recall any congressional antitrust investigations in the pre-BCS bowl system; is that what he wants? Given the system of conference tie-ins, teams from small conferences had little chance of playing in the big bowl games then either. Likewise, the effect of the BCS has been primarily to swap teams among the four bowl games in question to ensure a #1/#2 matchup, not to exclude teams from non-BCS conferences (who wouldn't have been playing in the Rose Bowl in the first place).
Even though the BCS system is reasonably open to qualified teams from non-BCS conferences, one thing that is worth checking is whether or not perceptions that is not have increased the disparity between BCS and non-BCS conferences. Below are the conference strengths of I-A conferences over the six years of the BCS agreement and the previous six years. To facilitate year-to-year comparisons, strengths have been adjusted to keep a constant average of 3.55 for teams in BCS conferences.
YEAR B10 P10 SEC B8/B12 ACC BE MWC USA/SWC WAC MAC SB/BW
2003 3 3.572 4 3.463 1 3.728 2 3.616 5 3.454 7 3.379 6 3.395 8 2.666 10 2.576 9 2.586 11 2.231
2002 4 3.468 1 3.683 3 3.563 2 3.663 5 3.455 6 3.426 7 2.761 8 2.674 10 2.334 9 2.481 11 2.260
2001 5 3.515 4 3.543 2 3.573 1 3.656 6 3.429 3 3.561 7 3.000 8 2.830 9 2.610 10 2.530 11 1.928
2000 5 3.477 1 3.681 4 3.581 2 3.649 6 3.292 3 3.592 7 2.983 8 2.972 9 2.774 10 2.496 11 2.480
1999 1 3.912 6 3.267 2 3.715 4 3.460 3 3.546 5 3.283 8 3.112 7 3.240 9 2.610 11 2.334 10 2.354
1998 2 3.656 3 3.529 4 3.497 1 3.794 6 3.363 5 3.383 8 2.878 7 2.896 8 2.878 11 2.155 10 2.451
1997 4 3.532 2 3.911 1 3.927 5 3.299 3 3.645 9 2.795 6 2.904 8 2.903 6 2.904 10 2.716 11 2.464
1996 1 3.820 2 3.740 4 3.458 3 3.626 6 3.268 5 3.292 8 2.851 7 3.142 8 2.851 10 2.315 11 2.154
1995 1 3.953 4 3.365 3 3.585 2 3.644 6 3.304 5 3.324 8 2.899 7 3.211 8 2.899 10 2.468 11 2.224
1994 1 3.856 3 3.629 2 3.636 4 3.429 7 3.322 8 3.321 5 3.331 9 3.108 5 3.331 11 2.032 10 2.221
1993 4 3.571 1 3.919 3 3.626 2 3.660 6 3.149 5 3.248 8 2.968 7 3.136 8 2.968 11 1.905 10 2.319
1992 6 3.330 1 3.838 3 3.596 5 3.433 2 3.682 4 3.436 7 3.126 9 3.119 7 3.126 10 2.483 11 2.433
BCS 3 3.600 4 3.528 2 3.610 1 3.640 6 3.423 5 3.437 7 3.022 8 2.880 9 2.630 10 2.430 11 2.284
prev 2 3.677 1 3.734 3 3.638 4 3.515 5 3.395 6 3.236 8 3.013 7 3.103 8 3.013 10 2.320 11 2.303
all 1 3.638 2 3.631 3 3.624 4 3.577 5 3.409 6 3.337 7 3.017 8 2.991 9 2.822 10 2.375 11 2.293The numbers in the table above are taken from my current rankings, so the 2003 values might change somewhat over the remainder of the season. Twice the makeup of Division I-A has changed significantly. Between 1995 and 1996, the SWC disbanded, with the four best teams joining the renamed Big Eight and the others joining the WAC and newly-created conference USA. Between 1998 and 1999, the MWC split from the WAC; for years 1998 and earlier the MWC average is set equal to the WAC average.
What should be clear is that only three times in the last 12 seasons has a non-BCS conference been one of the six strongest conferences: the WAC in 1994 and 1997, and the MWC in 2003. However, overall there is a clear gap between the six power conferences and the others; the difference between the #1 and #6 conferences exceeds the difference between the #6 and #7 conferences. The other question, has the gap increased as a result of the BCS, is harder to answer. On the surface it looks like the answer is "yes", but the SWC's strength before its breakup was largely due to modern BCS conference teams Texas and Texas A&M. Looking at the non-BCS columns with the most stable membership (MWC and MAC), it seems that the advent of the BCS has done little to change their strengths.
Of course, the BCS isn't interested in top-to-bottom strength shown above (I show it for the sake of stability); they only care about the best team from each conference. Below are the highest-ranked teams in each conference, loosely attempting to keep membership constant from year to year. (In other words, the Texas schools are counted in the B12 column, even during the SWC years.) The strength rating is taken from the median likelihood ranking, making the same adjustments as above to keep the average constant.
YEAR B10 P10 SEC B12 ACC BE MWC USA WAC MAC SB/BW
2003 5 4.088 3 4.313 2 4.393 1 5.256 11 3.996 10 4.012 14 3.863 22 3.695 20 3.762 9 4.033 46 3.347
2002 1 5.403 2 4.966 4 4.688 5 4.422 13 3.870 3 4.678 32 3.482 31 3.483 18 3.760 33 3.446 57 3.153
2001 12 4.064 2 4.493 3 4.490 4 4.348 10 4.070 1 5.673 18 3.765 17 3.773 23 3.638 21 3.658 61 3.119
2000 12 3.845 3 4.866 9 4.273 1 5.956 5 4.639 2 4.976 18 3.717 31 3.553 28 3.621 19 3.711 35 3.463
1999 4 4.607 18 3.711 8 4.349 2 4.828 1 5.665 3 4.630 31 3.473 15 3.791 49 3.243 6 4.388 50 3.243
1998 2 4.654 8 4.425 1 5.534 4 4.531 3 4.580 19 3.867 13 4.060 9 4.356 45 3.273 33 3.516 52 3.231
1997 2 5.447 7 4.550 4 4.917 1 5.632 3 5.089 35 3.456 18 3.877 19 3.715 45 3.298 30 3.528 71 2.975
1996 2 5.132 3 5.076 1 5.136 5 4.701 4 5.060 11 4.231 8 4.482 22 3.806 34 3.508 56 3.142 52 3.223
1995 4 4.688 15 3.973 2 5.152 1 5.780 8 4.298 14 4.000 38 3.426 17 3.872 56 3.217 11 4.122 50 3.279
1994 1 5.650 14 3.992 3 4.895 2 5.380 5 4.710 7 4.283 10 4.077 44 3.321 45 3.301 50 3.221 43 3.327
1993 6 4.496 8 4.414 2 5.036 3 4.997 1 5.126 7 4.423 41 3.306 16 3.926 32 3.392 65 2.953 54 3.148
1992 6 4.277 7 5.803 1 5.408 5 6.041 3 4.914 2 4.919 31 3.474 34 3.420 17 3.813 24 3.625 55 3.190
BCS 6 4.443 5 4.462 3 4.621 1 4.890 4 4.470 2 4.639 9 3.727 8 3.775 10 3.550 7 3.792 11 3.259
prev 3 4.948 5 4.635 2 5.091 1 5.422 4 4.866 6 4.219 7 3.774 8 3.677 10 3.421 9 3.432 11 3.190
all 3 4.696 5 4.548 2 4.856 1 5.156 4 4.668 6 4.429 7 3.750 8 3.726 10 3.486 9 3.612 11 3.225The gap between the big six conferences and the rest is even more apparent here. The average strength of a BCS conference champion is 4.429 or higher; that of a non-BCS conference champ is 3.750 or lower. Since the BCS includes eight teams, the key question is how frequently a conference's highest-ranked team is ranked #8 or better. The Big East has the worst track record (7/12), followed by the ACC and Pac 10 (9/12 each). This is not a huge surprise; smaller conferences have fewer teams and are thus less likely to have unusually good ones. Nevertheless all six major conferences contain a top-eight team over half of the time.
In stark contrast are the non-BCS conferences, which only twice (once pre-BCS, once during the BCS) have had a team ranked in the top eight. Interestingly, the well-publicized case (1998 Tulane) is not one of these. Tulane was unbeaten, but had played such an easy schedule (the only opponents with winning records were three 5-loss conference opponents) that they did not have a compelling case for BCS inclusion. The only top-eight team from a non-BCS conference during the past six years has been Marshall in 1999, which I ranked #8 before the bowl games. Marshall's downfall was that the voters watch the games and vote not only based on wins and losses, but how good the teams look. Because Marshall didn't play as well as might have been expected from a top-eight team, they were ranked #12 in the polls and thus left out of a BCS bowl. The only other case was BYU in 1996, although they were #10 in my pre-bowl ranking and moved into the top eight only after playing bowl games.
In summary, it seems clear that the BCS system is very reasonable, given the two very different levels of play within I-A football. While the conferences without BCS bids will be unhappy about their lack of privilege, it appears that the selection of the six conferences is quite fair. All six conferences produce teams ranked in the top eight at least half of the seasons I examined; none of the non-BCS conferences has produced a top-eight team more than once. Put differently, if one were to implement a playoff like I-AA, where about half of the conferences have automatic bids and half do not (and the decision was made on conference strength), the automatic bids would go to the BCS conferences.
More significantly, it does not appear that the presence of the BCS has hurt the level of play in non-BCS conferences. Only one such team in the 12-year study had a top-eight ranking prior to bowl games; that event happened during the BCS years. The rate at which top-twenty teams has been created has likewise been stable or even increased somewhat. Finally, the level of top-to-bottom play has been fairly constant from pre-BCS years to non-BCS years.
Despite his claim to the contrary, it seems the Tulane president and his cohorts are looking for a handout. The BCS system accurately reflects real differences in the quality of play between big and small conferences, and there is no rational reason why the BCS should include a conference that has less than a 1/10 chance of having a top-eight champion. What these teams want is a $10+ million freebie, where one or two of their teams will get a lucrative bowl bid and show the whole world that they are unable to compete. (One can only imagine a TCU vs. LSU matchup; from my current rankings I give LSU nearly a 90% chance of winning such a game.)
Note: if you use any of the facts, equations, or mathematical principles introduced here, you must give me credit.