That's tougher to argue, based on FEI statistics.
BYU's offense (ranked #47 in FBS) in particular has struggled to put up points (< 20 offensive points scored) against the 2 elite defenses they've played (Utah #14 and SMU #18).
BYU's defense has been borderline elite (#19). They've only given up > 30 points once this season, despite playing some good offenses (UCF #16 and KSU #27). They do seem to struggle though in a few key areas, such as finishing sacks or forcing punts when they get it to 3rd down.
BYU's special teams have been legitimately elite (#5!). They've scored 3 TDs this year, and Will Ferrin has proven to both have range (made FGs over 50 yards) and ice in his veins (game-winner over Utah). Still though, special teams generally matter more in close games, rather than blowouts.
Overall, FEI has BYU currently ranked #21 in the nation. I think that's a little low, but wins over Kansas (#40 overall), ASU (#30 overall), and Houston (#67 overall) would help BYU get into that top 15 overall ranking I think, based on FEI.