People don't want the game officiated via video and replay, otherwise the replay rule would be worded such to correct bad or missed calls.
That's not the role of replay and that's the reason this is bad. May as well just remove refs if that's the intent you're looking for.