If all it was was to say no to racial or sexual animus as a heightening factor, probably most people would agree. The problem is when you have people saying things like "having the drum and feather on a football helmet is hate speech because some native americans do not like it" - that's not hate speech. And as long as you can make that argument, everyone can make that argument about something, leaving you with really no category at all.
Personally, I'm fine arguing that "f** the Mormons" chant is hateful and wildly wrong. But hate speech is meant to invoke government prohibitions. I don't see a role for government here. I think it's fine for the University to hold itself accountable for that sort of thing.