What happened, is we discovered that files are really useful because you don't need to declare the format of data that goes into the file. So the operating system can handle things like reading and writing and the application can organise how it wants to keep the data in the file.
The same really is for sockets. It is really useful to have somebody transfer the data for you in a stream and you, the application, only worry about the format of the data.
Bytes are a "narrow waist" and in-fact DBs actually use our system for storage. By supporting bytes, anything that can be serialized can be stored by the next layer up and the contract is very simple.
Different types of data are legal in different jurisdictions (for example the definition of PII data), the physical location of the hard drive matters.
When medical data is stored, where and how is important. When handling data that needs to, legally, needs an audit trail, abstractions won't do.
When data is needed at low latency, the details matter. When cost is important (egress charges per operation or counted by size of data transfered), details matter.
Not exactly: what matters is the legal designation of the data storage device. The location of that device is one of many factors that "matter", but not to the application, or developer, or user. They only "matter" to the law. We aren't going to start writing UnitedStatesFileWrite() functions, now, are we?
Instead of considering the physical location of a hard drive, what we should be doing is querying a data storage object which has the properties we want:
io_construct = DataStorage()
storage_search = io_construct.DataStorageSearch({
"contains": [
{ "legal": {
"jurisdiction": {
"location": [ {
"country": "US",
"state": "California"
} ]
}
}
},
{ "record": [ { "email": "[email protected]" } ]
}
]
})
with io_object as io_construct.AttachDataStorage(device = storage_search):
io_object.read()
1. Price
2. Brand of whoever is providing the storage (matters because it's a proxy for lots of other details)
3. General physical location
Once you made those decisions services like S3 abstract the rest. There are tools that let you access these via FUSE (in which case client protocols don't matter).
What we really want is to store some data somewhere, and later be able to retrieve it, without necessarily knowing what it was we stored or where or how. And we don't want to think about what server it's on, or what hard drive, or what folder. And we don't want to think about client protocols or query languages.
All of that would be possible if we reinvented i/o. Basically, just imagine what you want your experience to be, and then start making up names for functions that do that. Stuff that in a kernel, or a standard library. Now you have i/o that's based on how you really want to use data. The backend implementation of it can vary, but the point is to make the user experience what we actually want rather than what somebody else thinks is practical. Make the data interface you want to use, and make it a standard.