Playing more Nickel robs us of an LB.
Playing a safety in man coverage limits our blitz potential and makes us more at risk in over over-the-top if its not the doubled receiver. Also, if the doubled receiver beats both of them, its a TD, as there'd only be 1 safety to get him, and he's likely on the other side of the field.