It isn't that we are using up our best athletes in other sports. We statistically should have plenty of athletes to go around.

It is our culture. We don't care as a nation about soccer. It is still a niche sport (growing yes, but not capturing the country's hearts). That will probably never change and because of it, there is likely a plateau as to how good we can be at the sport.