Not having enough muscle mass significantly increases your risk for injuries and debilitation as you're older.
So some balance probably makes sense. For me, I try to not eat red meat most days (probably a couple times a week), I eat almost no processed meat (that's the worst kind for cancer), eat more fish than I used to (salmon is really healthy if sourced well. Expensive though, but worth it), and supplement protein in other ways.