I would go in a heartbeat. If you have that big an issue wear the other teams gear and cheer for them, or just cheer for the other team.
The Rose Bowl is an event. Having Utah there could make it even more fun...you have a bigger interest in the game will cheer for Penn St/OSU/MN/WI etc.