It was fine even when the Occupy Wall Street encampment was down in Herman Plaza in the Fed Reserve District. Going back to the early 80s at least, there was always a sizable homeless population that would barricade themselves into store fronts with trash bags at night. But right at the end of 2012, I noticed things starting to turn. There were always vagrants on every street corner downtown, but I started seeing the crap on the sidewalk. And the homeless got more aggressive (they have always been aggressive, but it got worse).
My wife came out for a long weekend and a guy asked me for change. I told him sorry, I do not carry cash. He said, "how would you like me to blow you away with a sawed off shotgun" and turned away. He was actually wearing a trench coat. I don't react well to stuff like that and cut him off as he turned away and asked him to repeat what he said. He turned again and walked away.
I love San Francisco. It is sad what it has become.