difference between the US and most other countries is that we rarely walk to get around town, whereas in most other countries you walk a lot. We tend to drive everywhere, since our cities are mostly not very dense and we don't use public transportation very much, while in most other countries most people are much more likely to walk around their densely populated cities, and to walk and climb stairs to/from subways, buses, trains, underpasses, overpasses, etc. We just aren't as physically active in our daily, routine, non-workout lives as most other countries are.

Like you said, there are also dietary differences, of course, especially in places like Japan. But when I compare the US to other countries with similar diets (say, Canada, Australia, Mexico, and most of Western Europe), we walk much less than they do, and are also heavier, for the most part.

Genetics also matter a lot, as does age.