What's the reason, then? Looking at Appalachia and southern Italy, I don't see any reason to believe that ethnic/racial/cultural homogeneity is a magic bullet.

It's definitely not, proven by the examples you cited.

Where you get ultra-low crime is when you combine that homogeneity with good public education. Take a look at the education systems and levels in places like Japan or Norway, and then compare to Appalachia or Sicily or various middle eastern nations. Also look at the levels of religiosity.

But if good public education and low religiosity were the magic bullets, the Soviet Union would never have fallen; if homogeneity was necessary, Singapore wouldn't work and the Ottoman Empire would always have been dysfunctional; if high religiosity was bad in itself, Bavaria would be a hellhole. Some religions get in the way, others don't -- and some actively help to enrich their societies. (Zoroastrianism should come to mind. Rice was the staple crop of Persia before it was the staple of China, thanks to pious Zoroastrian agricultural research; and building an aqueduct or draining a swamp was a fast path to paradise. Down to the present, the Zoroastrians do pretty well for themselves -- the Parsis of India are the main population of Zoroastrians today -- and some of their habits of mind seem to still be present in Iran today.)

I've come to believe that the potential to rise in social position is what matters most, except possibly in Zoroastrian societies. Appalachia, Sicily (including lower-class parts of Boston), and the Middle East all agree that where you were born is where you'll stay, and that ambition is ridiculous or contemptible; while the healthiest societies are the ones where talent can bubble to the top even if you're not the judge's son.

My favorite example of the vitality that social mobility brings is the Ibo of Nigeria -- "the Jews of Africa", the most successful and energetic of West African peoples. Also look at the dynamic nature of the early Islamic world, compared compared to the stagnant nature of the modern one. And what's true of the Islamic world is equally true of the Iberian one; Brazil and West Virginia have the same basic problem. And back in the Middle Ages, too, it was the most fluid societies -- Lombardy, England, the Low Countries, and the Iberian states -- that were the most successful ones.

You make some great points, but for your Soviet Union point I'd like to counter that indoctrination does not equal "good public education". Good public education teaches you to question authority and to think critically. That kind of education is notably absent in authoritarian regimes.

Also, I never said that religiosity by itself would result in downfall of a society. Lots of religious societies have done well in history, but they also had relatively good education. The early Islamic world you mention I think qualifies here. It's when you combine religiosity with terrible education that you're really doomed. The education serves as a check against the religiosity getting out of control and turning into theocracy, or whatever you call the wackiness we're seeing in the right-wing here in America today.

The Soviets really did do a good job in some important areas. No one had taken the basic skills, the "three Rs", seriously in Russia before them; science and engineering were also pretty good (apart from Lysenko in biology), and while 20th-century culture was closed off, 19th-century was certainly accessible. (Hugo and Dumas were a big part of Soviet popular culture, not in the sense of "what the state sponsored" but in the sense of "what people actually read.") Insofar as critical thinking is a subset of logic, science and math taught at least a little of it, too. Understand that I'm not defending the Soviets as a whole -- I think they were mostly cancer, and that the historical record backs me up on that -- but deserve credit where credit's due.

I think that the American right is a unique sort of society, and that patterns drawn from their behavior can't be relied on in most other contexts. Their religions look like Christianity but act like non-syncretistic paganism ("worshiping oneself in the guise of worshiping God"), and their hostility to an awful lot of scholarship seems to originate in not wanting to be taken for a ride -- and in the confidence of many peoples, especially relatively backwards ones, that they already know everything that's necessary in order to live well. (The Persians told Chardin as much in the 1500s, as mentioned in The Structures of Everyday Life.)

Have you read Albion's Seed? If not (and it's about 600 pages of historical scholarship, so I won't blame you), look up Slate Star Codex's review, which makes the book's essential points in reduced (and maybe slightly caricatured) form.

Pretty much all the crime in rural areas is a side effect of drugs. There isn't enough anonymity in those areas to actually be a career criminal (other than dealing drugs).

These societies' problems are certainly not recent, and they're not just due to drugs. For Appalachia, read _Albion's Seed_ or at least Slate Star Codex's review ("the sensibility is positively Orcish"); for Sicily, read Peter Turchin's _War and Peace and War_ (watch for the section on "asabiyya black holes"), and probably _The Moral Basis of a Backwards Society_. Both of these societies have been absolutely awful for centuries.

Singapore is one of the most diverse countries in Asia and has similar crime rates to Japan.

It depends what you mean by "diverse". You could say Switzerland is diverse, they speak 3 languages there, but there is an essential Swissness that everyone generally agrees on. I believe the same is true of Japan. Maybe the word I actually wanted was somewhere between homogeneous and harmonious.

Over 15% of Singapore's population are non-citizens ("foreign talent"), so whatever they're doing it's not the same as Switzerland

Singapore is over 75% ethnically Chinese. That's still remarkably homogeneous by US standards.

Singapore has similarly low crime figures and is not remotely as homogeneous as Japan.

You could say Japan committed quite a lot of crime 75 years ago, but the population hasn't really changed since then.

Probably, these days, everyone has something better to do.

People often claim this, but I have never seen evidence for causation--just theories, whereas the actual evidence is only correlation.