If you can hear the conversation even if you aren't a party to it. Most prosecutors have come down on the side of if you can hear it you can record as long as you aren't concealing your presence.
This came up because someone recorded a legislator and lobbyist conversation through a closed door.