The rivalry game is just too fun to not have both schools in the same conference competing all the time. It's also really good for fan engagement on both sides.
Furthermore, I have every bit of confidence that BYU will start to come out on top regularly in football (already doing it in basketball for the most part) within a few years now that the Pac12 advantage has been neutralized. Utah's best years are behind them imo, while BYU is just getting started.