The way it was explained to me by black teammates in high school was: Just like you or a sibling may say something critical about your mom (even if u shouldn't), if someone else talks about your mom, there's going to be trouble.
Black people took control of the word to take their power back, etc, and can say it to each other if they want to and can allow certain other minorities to use it if they so choose to let them, because historically they didn't own black American slaves and may have some other shared experiences in America as well.