But the other team will still score and score a lot. 98% of points are scored 5 on 5 in basketball.
Keeping 4 guys back on defense in basketball will only serve to prevent you from scoring and will make sure you don't give up any fast breaks. But the other team will still score plenty in their offensive set.