With Viaweb, the stores were islands, each dealing exclusively (or almost exclusively) with its own dataset. They would not run into locking/contention issues as they scaled the way some other apps would if they used flat files and conventional locking.

The database landscape was different in 1995. There were no good free relational databases, and no cheap Oracle licenses.

Lisp makes it very easy to write out any data structure. (Most Scheme implementations can only write non-circular structures easily, but Viaweb used CL). So if your app is amenable to using flat files, coding storage/retrieval is trivial.

So basically, if you're writing an app with lots of separate datasets, flat files are a viable option. If you're using Lisp, they're an easy option. If it's 1995, they're a cheap option.

It is interesting how flat files can manage the evolution of the data structures you use? Any advice?

Use hashtable for everything. So Person who has a name address age and occupation is represented as a hashtable with those fields. When you want to add a new field like emailaddress it's just another key in the hashtable.

Ideally you want to code the person accessors so that they still specific externally

person-name person-age

the rest of the program uses these so it becomes easy to move person to a different structure if that's what you want .

You want to write some function to read a hash-table and write on to disc. In lisp this is trivial, but most languages have some sort of serializer. Then a function to load all the hash tables up when you start up.

If I understand your question you're wondering how to deal with older flat files when your app expects a newer data structure.

If you're using s-exprs make your structure a list where the first element is a version identifier. If you aren't using s-exprs then use some other extensible format like XML.

Great points, thanks ;)