I've also had. To be fair, the two pairs of Xero's I've had have been the Prio which is much more of a road/casual shoe than the Altra Lone Peaks I've had.
The Xero's are super comfy, but I've never really worn them for hiking/running, which is what I've used the Lone Peaks for. I've really only worn them for casual/work shoes. I walk a lot for work, but they would be pretty light/insubstantial for me if I were to wear them camping/hiking/outdoors stuff.
They do have other kinds that are more for trail/off-road situations though.