I don't think it's Sunday games either, I think it's more like time/money commitment to be able to go to games for both. I look at a lot of NFL cities that have college football in the same city or area and those programs generally struggle with attendance and support unless they have a really good season. It's why the Pac-12 struggled with interest, a lot of those schools share a market with an NFL team and are dwarfed in comparison.
Baseball will do well, if for no other reason than local fans coming to see their favorite teams come here to play.