Lars was always the weak spot. Cliff told Scott when the Ride the Lightning tour was over, Lars was out. After Cliff's passing, Scott asked if they were still going to dump Lars and he said he couldn't bear to cut ties with another bandmate after Cliff dying.
Being a founding member isn't always enough to stay in a band. Look at Sepultura.