If BYU had substituted UCLA, Michigan and Missouri with Alabama, Clemson and Oklahoma, the end of year record would have been the same, but the SOS would have been much, much better. Likewise, replacing Wagner with a second Fresno State would have improved the SOS without risking another loss.
Likewise, with just a little increase in the SOS, BYU might have had a couple of more losses this year, depending on which team was substituted out and for whom.
The 2003 and 2004 teams played tough SOS, but I think the record would have been the same if the really great teams that pushed the SOS upward had been replaced by good but not great teams that BYU would have lost to anyway.