I went into a convenience store deep in the woods of North Georgia and upon hearing my strange accent the woman at the cash register said (with suspicion in her voice) "You're not from around here, are you?" I then proceeded to explain that I was born in Utah and that the Yankee Army invaded my home state in 1857/1858, several years prior to when General Sherman invaded Georgia. That mollified her as she discovered that she had found a strange but kindred spirit. She then explained to me in quite colorful language the problems she had with "Yankees." Now, if she had determined that I was a blankety-blank Yankee, she told me that she was prepared to tell me to LEAVE!
So, on the one hand we can be told that if we don't like it, why don't we leave... Conversely, we can also be told to leave, whether we like it or not.