Navigate's College Football Playoff Predictions (Part I, II, III & IV)

AthleticDirectorU has partnered with Navigate to present a rich prediction model – based on 3,500 simulations of the rest of the season –  for this season’s College Football Playoff.

Part IV  (12/20)

THE 4TH SPOT

As soon as Clemson took a commanding lead on Notre Dame in the ACC Championship, it was pretty clear that Alabama (regardless of what happened against Florida later in the night), Ohio State, and Clemson were going to get in the CFP. And the 4th spot was going to come down to Notre Dame and Texas A&M.

As Clemson’s lead grew, we tweeted that a loss by more than 30 points could cost Notre Dame a CFP berth. Why? In Navigate’s CFP prediction model, teams get additional punishment for losing by more than 20, and the punishment increases as the size of the loss increases. This is how our model is most accurate in reflecting the CFP committee’s decision-making in the past, and it tells us they start to care about losses in excess of 20 points, and the magnitude of the loss above that threshold matters.

In the end, Notre Dame’s 34-10 loss was a bad one, but 24 points wasn’t quite bad enough to knock them below Texas A&M in our model – or, as it turned out, in the CFP committee’s rankings. Notre Dame’s loss needed to be at least a few points worse than the Aggies’ 28-point loss to Alabama. Or in other words, if both contenders were going to have one loss to another CFP top 4 team, Notre Dame’s loss needed to be embarrassing, not just bad. It needed to make the committee re-think whether it could justify keeping Notre Dame in after a loss of a certain magnitude. It didn’t quite get there and the Fighting Irish edged out the Aggies for the final spot.

THINKING ABOUT EXPANSION

Our CFP prediction model is built to predict the top 4 because as of now, that’s all that matters. It was 24-for-24 entering this year, and after we made some assumptions to adjust it for 2020, we ended up getting the top 4 right again. The reason we’re able to maintain this accuracy is that the committee has been fairly consistent over the years in the way it parses out who makes it and who doesn’t among the final contenders. But after the top 4 … it gets dicey. 

We don’t want to read too much into this year’s results beyond the top 4 because this was a bizarre year. Truncated schedules, cancelled games, and teams that had star players frequently missing all put a lot of extra considerations into the committee’s decision-making, some of which were undoubtedly not captured in our model. But here is what we’re sure of: As much as there is some debate right now about the final CFP selections, it will grow exponentially with expansion. 

Right now, it essentially comes down to the 0- and 1-loss Power 5 teams, and only if that group is weak does a 2-loss team have any chance. If the playoff expands – or maybe we should say when the playoff expands, as it seems inevitable – to 8 or 12 teams, there will likely be 5 automatic qualifiers from the Power 5, possibly 1 spot reserved for the Group of 5 … and then the remaining 2 or 6 spots will come down to a much larger group of 2 (or 3?) loss teams. Creating consistent logic to parse out who is deserving among those teams will be extremely difficult for the committee, and extremely difficult for Navigate to model. 

THE TROUBLE WITH FLORIDA

Exhibit A to the point above is Florida. Historically, a team that has a late loss to a bad team is going to take a massive fall in the rankings – especially if it’s a 2nd loss. But Florida only fell from 6th to 7th in the CFP rankings after its loss to a 3-5 LSU last week. In our model, the Gators fell to 11th because LSU – despite winning the national title last year – had fallen so far in our model with some truly bad losses this year, and the model no longer cared what they did in 2019. But it seems clear the committee did not consider it an awful loss and the committee didn’t particularly care that Florida lost again in the SEC championship because they kept it close against No. 1 Alabama. In our model, a 3rd loss is death because our aim is to make sure a team with 3 losses is not making the top 4 – because those 4 spots are all we care about. But Florida finishing 7th in the final CFP rankings is further proof that when the playoff expands, our model will need to change drastically beyond how it chooses the top 4.

BACK TO THE PRESENT, BAD TIMES FOR THE GROUP OF 5

While the playoff remains at 4 teams, Cincinnati reaffirmed what we had concluded based on our model in the past: There is an extremely narrow path for a Group of 5 to get in and it didn’t happen this year. We printed this before, but as a reminder, here is what a Group of 5 team needs:

• An undefeated season 

• At least one win over an above average Power 5 school

• A fairly strong season for the Group of 5 school’s conference (e.g. at least one or two other schools in the conference also beat a Power 5 school)

• At least one Power 5 conference, if not two, producing a relatively weak champion

Without all of those ingredients – Cincinnati didn’t play a Power 5 team this year – the committee was never going to seriously consider the Bearcats. Our model ranked them 8th, as did the CFP.  

FINAL THOUGHTS ON THE REST OF OUR TOP 10

When we have applied this model to client work, we have never worried much about the rest of the rankings. Again, the top 4 is all that matters. But in this case, since we’re sharing a Top 10 with you, we thought it made sense to explain Indiana and USC.

Indiana finished 7th in our model because it only had 1 loss, which came against the CFP-bound Buckeyes, and the Hoosiers had a trio of strong wins. When they beat Penn State to open the season, the Nittany Lions were a consensus Top-10 team and highly ranked in our model. When Indiana beat Michigan and Wisconsin, both of those schools were also ranked. To put it simply, if a school has three wins against teams that were ranked at the time, and only has 1 loss to one of the top teams in the country, our model is going to rank them pretty highly. Of course, our model’s only punishment for truncated schedules applied to removing the usual title game bump that the Big Ten and Pac-12 would have received. Considering the committee put Indiana 11th in its final rankings, it seems clear there was some extra punishment for the Hoosiers only playing 7 games. 

Along those same lines, USC finished 10th in the Navigate model because a Power 5 team that only has 1 loss is inherently going to finish fairly high. But the Trojans’ lack of quality wins and 6-game schedule led to an understandable 17th final ranking by the CFP committee. 

If we put an additional component into our model that punishes teams for a smaller number of games played, we could likely end up getting Indiana and USC closer to their final CFP spots. But we could also flirt with Ohio State (having only played 6 games) falling out of the top 4. In the end, considering the top 4 is all we cared about, we sacrificed tightening our margin of error later in the rankings to ensure we had confidence our model would reflect the committee’s CFP semifinalist selections. 

Part III  (12/13)

HAS ANYONE LOCKED UP A CFP SPOT?

• Yes, according to our model, Notre Dame and Alabama can both lose in the conference championship games and no scenario has them falling out of the CFP.

WHAT IF THE FAVORITES WIN?

If Alabama, Notre Dame, and Ohio State win … our model says Texas A&M edges out the Big 12 champion for the final spot. Why? Well, two cancellations from this past weekend had an impact:

1. Because Cincinnati’s game against Tulsa was canceled, the Bearcats missed a chance to improve its resume before potentially beating Tulsa again in the conference title game. Our model now thinks Cincinnati has no chance to get selected if the committee’s past behavior is any indication. (For the record, even if Cincinnati had beaten Tulsa this past weekend, its chances were continuing to diminish to the point of needing complete chaos these last couple weekends. But now the model sees no path in.)
2. Because Oklahoma’s game against West Virginia was canceled, the Sooners lost a chance to increase its rating in our model just a little bit more before the Big 12 championship game. And that had HUGE ramifications on the projections with the razor thin margins in the competition for the final spot. Previously, our model projected Oklahoma beating West Virginia the vast majority of the time, which meant a slightly higher rating for the Sooners entering the title game, which meant Oklahoma or Iowa State (if it beat a slightly stronger Sooners team) would be a virtual toss up with Texas A&M. Now, if Alabama, Notre Dame, and Ohio State win, our model has Texas A&M snagging the 4th CFP spot (assuming it beats Tennessee) a little more comfortably over the Big 12 champion (regardless of whether it’s Oklahoma or Iowa State). 

Now, you might say: Wait, of the previous 24 teams to be selected, only 2 (2016 Ohio State and 2017 Alabama) did not win their conference championship, which seems to favor the Big 12 champ over Texas A&M. But the committee has NEVER selected a team with two losses. If this scenario plays out, it’s hard to imagine the committee choosing either a two-loss Oklahoma (with one of those losses to Kansas State) or a two-loss Iowa State (with one of those losses to Louisiana) over an Aggies team that only lost to Alabama.

LET’S TALK UPSETS

• If Clemson beats Notre Dame … they are both in. 
• If Florida beats Alabama … as already noted, Alabama is still in, but Florida has no scenario getting them in.
• If Northwestern beats Ohio State … Ohio State and the Big Ten are out.

HOW DOES THE BIG 12 CHAMPION GET IN?

• The Big 12 champ needs any TWO of the following three results to happen:
  • Notre Dame beats Clemson 
  • Tennessee upsets Texas A&M 
  • Northwestern upsets Ohio State 

WHY CAN OHIO STATE GET IN AND USC CANNOT?

• As a reminder, our model is NOT based on what we think is fair or right or reflective of our opinion in any way. It is solely built to reflect how college football voters and the CFP committee have behaved – and in 2020, we have made some tweaks based on comments from the committee and our hypotheses around how they will behave in this unusual year. With all of that said, two factors – one that has nothing to do with 2020, one that does – hurt USC, but do not hurt Ohio State:

• 1. Our model has always been most accurate when we take previous seasons into account. Or in other words, every team doesn’t start the season on a level playing field because rankings are anchored to expectations or reputations (or both). In this case, Ohio State started near the very top of the rankings (2nd) and USC started 18th. Even with its truncated schedule, Ohio State has stayed in the top 4 and USC has not had enough strong wins to climb above the other teams still in contention. 
  2. It has been apparent all along that a truncated schedule – or fewer data points, as is often said – matters to the committee. With that in mind, we removed the bonus points that a Big Ten or Pac-12 champion would normally get because they’re playing so many fewer games than the ACC, Big 12, and SEC. That change to the model hurts Ohio State’s seeding, but it is not enough to knock the Buckeyes out of the top 4, whereas it prevents USC from breaking in. If the committee did give the Trojans the same credit for winning its conference title as it gives the ACC, Big 12, or SEC champion, USC could crack the top 4. But if our assumption is correct and the committee does NOT give equal credit, the Trojans can only get as high as 6th. 

WHAT ARE THE CHANCES IN THE P5 CHAMPIONSHIP GAMES?

• According to our model …
  • ACC: Notre Dame beats Clemson 57% of the time
  • Big 12: Oklahoma beats Iowa State 51% of the time
  • Big Ten: Ohio State beats Northwestern 85% of the time
  • Pac-12: USC beats Washington 81% of the time
  • SEC: Alabama beats Florida 81% of the time

Part II  (12/6)

WHAT HAPPENED TO OKLAHOMA AND IOWA STATE?!

Oklahoma’s 27-14 win over Baylor and Iowa State’s 42-6 win over West Virginia might not seem like much, but along with Oklahoma State’s loss to TCU, we now know that Oklahoma is in the Big 12 title game against Iowa State – and in a bizarre season where the battle for the 4th CFP spot will be tight, that has a big impact on the probabilities.

Our model thinks the Big 12 champion has a 38% chance of getting in – the same as last week – but now instead of assuming that’s Iowa State more often than not (because last week it assumed Oklahoma could still miss the conference championship game), it now favors Oklahoma. Why? Despite Iowa State’s early October win over Oklahoma, our model has Oklahoma rated higher and projects the Sooners to win the Big 12 title game 55% of the time. Do we have concerns with this swing in the percentages for these schools individually? Yes, a little. As a reminder, our model is meant to reflect what the CFP committee will do at the end of the season, and we know the committee currently has Iowa State rated higher. But the combined probability of the Big 12 champion getting in (38%) has not changed, so we would not be too worried if we were Iowa State fans. This is probably just a quirk in our 2020 model, which we’ve had to adjust for a quirky season.

TEXAS A&M MOVES UP

By virtue of a quality win over Auburn, Texas A&M strengthened its case for the CFP’s 4th spot if Alabama, Notre Dame, and Ohio State win out and get in. In that event, the committee could be choosing between an undefeated Cincinnati, a 2-loss Clemson team, a 2-loss Big 12 champ, and a 1-loss Aggies team that will have been idle while the championship games are played. With that only loss to Alabama, our model sees Texas A&M’s chances rising to 32% now.

WHAT IF FLORDIA BEATS ALABAMA?

Our model thinks Florida could beat Alabama in the SEC title game 31% of the time, and if that were to happen, it predicts both Florida and Alabama to make the CFP in 79% of those simulations. The danger for Alabama would be if Clemson upsets Notre Dame and the committee picks both Notre Dame and Clemson in addition to Florida and Ohio State.

When you add in Texas A&M’s chances, our model now predicts the SEC has a 51% chance of putting two teams in the CFP.

CINCINNATI STAYS PUT

With two remaining games against Tulsa, our model thinks Cincinnati goes undefeated 51% of the time, and in those simulations, it makes the playoff 33% of the time.

HOW TO PROJECT OHIO STATE

As of now, our model assumes Ohio State still plays Michigan this upcoming weekend and still plays in the Big Ten title game. Even if Ohio State/Michigan gets canceled, our model has Ohio State rated so highly that we would need to add a new component that punishes a lack of games to keep Ohio State out. But … we’ll see what happens over the next week and decide if something like that seems necessary.

Part I  (11/29)

FIRST, THE DISCLAIMERS 

• These are NOT rankings. They are the projected chances of each school making the CFP once the regular season is completed and the conference championship games have been played. 

• Our CFP Projection Model works by inputting actual results – currently through Nov. 29 – and then running 5,000 simulations of the rest of the 2020 season to determine the likelihood each school is selected based on how the CFP committee has made its decisions in the past. 

• Our standard model is 24-for-24 in identifying the CFP’s top 4 after the conference championship games. However, we have made a few initial tweaks to account for the oddities of 2020 – for example, a penalty for playing less than 7 regular season games – and we might make additional updates based on the CFP committee’s rankings and comments.

• These results do NOT reflect whether we think a school is good or bad or deserving of a CFP selection. Our opinion is NOT reflected in these predictions and is completely moot. As stated above – and worth repeating – our model’s sole purpose is to reflect how the CFP committee has made selections in the past (and how they seem to be treating 2020) in order to predict the future.

WITH THAT OUT OF THE WAY, ON TO THE CURRENT PROJECTIONS

• There is a Clear Big 3. Unsurprisingly – and reassuringly for the architects of our model – there appear to be three schools who should make the CFP, barring any major upsets: Alabama, Ohio State, and Notre Dame. Even if Notre Dame were to lose to Clemson in the ACC title game, our model predicts BOTH Notre Dame and Clemson would be selected for the CFP 21% of the time. 

• The SEC Remains No. 1. Our model projects the SEC to put two teams in the CFP more often than the ACC (30% to 21%). The reason: Besides Alabama, the SEC has two strong contenders in both Florida and Texas A&M, whereas there is a major drop-off in the ACC after Notre Dame and Clemson.

• The Pac-12 is Likely Out. Only USC has any chance, and it’s not a strong one at 5%. Essentially, USC must finish undefeated and there needs to be incredible chaos for the committee to consider them for the final spot.

• The Big 12 is Still Alive. Iowa State has a 24% chance of getting in and Oklahoma has a 14% chance. Essentially, if the second-best team from the ACC and SEC do not finish the season strongly, and the Pac-12 champion is indeed punished for a severely truncated schedule, the decision could come down to the Big 12 champion or Cincinnati if it runs the table. It’s not extremely likely to happen for the Big 12, but our model does put them in the CFP 38% of the time.

• If the Group of 5 is Ever Getting In, This is the Year. Cincinnati has an 18% chance according to our model, although when we dig deeper, it’s a 34% chance if Cincinnati goes undefeated. Our model only has the Bearcats running the table 55% of the time, though, because they will likely need to face a fairly good Tulsa team twice – first at the end of the regular season, then in the AAC title game.
  • In a normal year, our model has told us that the only recipe for a Group of 5 school to get into the CFP would be the following ALL happening:
    • An undefeated season 
    • At least one win over an above average Power 5 school 
    • A fairly strong season for the Group of 5 school’s conference (e.g. at least one or two other schools in the conference also beat a Power 5 school)
    • At least one Power 5 conference, if not two, producing a relatively weak champion
  • UCF has come the closest of any Group of 5 school by putting together undefeated seasons in 2017 and 2018, with Power 5 wins in both of those years (albeit against Power 5 schools that were not ranked). But UCF was ranked 12th and 8th by the committee in those years, respectively (our model had them ranked 10th and 8th), which only reaffirmed that numerous ingredients are necessary for a Group of 5 school to even come close to the CFP’s top 4 spots.
  • Because Cincinnati’s schedule does not have a Power 5 opponent this year and the AAC isn’t having an exceptionally strong year relative to norms, our model would normally not see a way for Cincinnati to get in. But this is where we made a couple of 2020 adjustments. 
  • With the committee clearly punishing truncated schedules and NOT punishing the Group of 5 as harshly as normal years, we added a penalty for shorter schedules (as mentioned earlier) and softened the penalty that we use to separate the Power 5 from the Group of 5. And yes, our standard model – the one that is 24-for-24 in predicting the CFP – is most accurate when we have a factor that pushes the Group of 5 scores down.

• BYU Needs More Power 5 Games. As of now, our model has BYU going undefeated 89% of the time and never making the playoff. Its average final CFP ranking is 9th. It would likely require a win over a top Power 5 team, or two wins over middle-of-the-road Power 5 teams to have any shot at bridging that gap from 9th to 4th. 

• Some Long Shots Are Alive. A few Big Ten teams – Northwestern (5%), Indiana (4%), and Iowa (2%) – need to run the table and root for complete chaos to get in. It’s highly, highly improbable, but not impossible. And in that same vein, there’s still a 3% chance for Miami (FL) to find a way in.