I don't know how his donations break out from the athletics as a whole. For all I know, he's poured loads of money into the collective. I do know he's put a tremendous amount of funds and efforts in buildings and programs at BYU. That doesn't even account for what he's done on the temple side of the donations.
It's his money to give, no complaint there, but at least with buildings and programs, you leave behind a legacy.