Because the defense reads who's on the line and who can be in pass patterns.
You see a TE on the line with a receiver on his side. You expect him to stay and block and you shouldn't have to cover him. If it doesn't get noticed by an official it can be a really bad outcome for the defense.