your education, life, and perspective far more than a small hit to a GPA just because the classes are hard.
If you care only about GPA, avoid these classes. But if you do, you are missing out. I can't count how often I hear Jim Kearl pop up in my head throughout my career.