not right. CV has been shown in multiple settings in multiple countries to spread more quickly than influenza and causes severe disease in a significant enough number of people that hospitals get fully slammed with patients. If we had just gone on business as usual and done nothing about the virus, I think a lot more places in America would look like New York, New Orleans, Italy, etc. than they currently do. It definitely would impact the healthcare system, and the number of people being sidelined from work as well as the number dead would have a pretty big economic impact.
Do we need to panic, no. Do we need to shut everything down for 3 straight months, no. But, the argument you've presented is more akin to the "herd immunity" argument I've heard, which has been discussed as potentially having devastating health and economic impacts. It's more of the opposite extreme of the "close everything for months" argument - both lead to bad places.