This season Kansas beat BYU. BYU beat Baylor. Baylor beat Kansas. Imagine all these teams ended up with the same record. Head to head doesn't help. Or you have a situation where a team like Northern Illinois beats Notre Dame. Had Northern Illinois gone undefeated should they be ranked higher than Notre Dame and in the playoff? What about if they had one close loss? It gets messy really quickly and you end up with multiple teams with very similar resumes and you need some way to differentiate. I would prefer this be done using some sort of objective measure rather than the eye test.

I'm not arguing the model should have the final word. It's not even clear to me how much the CFP committee uses models in their decision making process. They certainly aren't just going with the top 12 according to FPI and putting those teams in the playoff. On Dec 8 Alabama was ranked #4 in FPI but clearly that didn't carry enough weight to put them in.

I think the main issue is there is an unsettled debate among fans about what the ranking criteria in this sport should be: the most deserving teams highest or the best teams. It's relatively easy to model (albeit imperfectly) the best teams if your definition of best is if team A would beat team B >50 percent of the time, team A is the better team. It's very difficult to model most-deserving because most-deserving is so subjective. Especially in scenarios like I outlined above where everyone beats everyone and everyone has the same record. This will be increasingly common with the new huge conferences.

So I agree, in a vacuum head to head should carry huge weight, maybe even most of it. But because there are 134 teams and a 12 game regular season, it rarely comes out that cleanly.