Common as well in first-millenium middle-east, as in Petra[0] or Mada'in Saleh[1], and there were quite a few cliff-dwelling people in precolumbian north-america, the most well known probably being the Anasazi which had well-known under-outcropping dwellings (e.g. Cliff Palace) but also actual cave dwellings like Tsankwawi[2] or very-close-to-cave dwellings like the House of Many Windows[3].

[0] http://en.wikipedia.org/wiki/Petra

[1] http://en.wikipedia.org/wiki/Mada%27in_Saleh

[2] http://en.wikipedia.org/wiki/Tsankawi

[3] http://en.wikipedia.org/wiki/File:House_of_Many_Windows.JPG

also just dig straight down. http://www.traveladventures.org/continents/africa/bet-giyorg...

In Granada, Spain, there are a bunch of such "cave dwellings" in the hill above the Alhambra Palace, from the inside looking quite similar to the photo in this story, small rooms with lime whitewash on the walls. I was graciously invited for an evening in one of those places, and while it was a bit cramped, and maybe a bit roughly furnished (you must use an out-house in most cases, if you were wondering, and the plumbing and electricity can be a challenge), it was agreeable and the temperature stays in a comfortable range in temperate climates.

Here's an interesting and brief travel documentary featuring some info on the cave dwellings in the second half:

http://www.youtube.com/watch?v=iMdeBviSO4I

And though I hate to link to the Daily Fail, they do have a nice little overview guide of Granada with a good picture of the cave dwellings from the outside and a bit of historical info here:

http://www.dailymail.co.uk/travel/article-569192/Granada-Des...

P.S. Go see Granada!

On a much smaller scale (one house), but I went for a walk here today:

http://maps.google.com/maps?q=Costozza,+Longare,+Italia&...

BTW, spinning the view around gives a nice view of the area, which is quite pretty.