How we ended up with an active device as our primary means of removable storage is me a puzzle...

We chose flash memory. It's a storage medium that involves a lot of tiny wires laid out in patterns going to the storage cell. We want to access these with only a few macroscopic wires connecting to a computer. This process has to be managed by a microprocessor of some sort.

Would you expect the host device to know the internal layout properties and hardware limitations of a purely passive bank of flash memory (e.g. when to mark a sector as failed)? It's possible but distributing the hardware and software to do this effectively would be a very difficult problem, and industry standardisation to mitigate it would be hard to arrive at as well, and deploying advances in flash memory would be hampered (remember when people worried about 720kb/1.44mb floppies? Imagine that for each new generation of flash memory. Is your flash drive slot MLC NAND flash capable? NOR flash? TLC? HKMG?)

Instead we live in a world where not just thumb drives, but even memory sticks and SD cards, are active devices. (This is, of course, to say nothing of the convenience of using one port for everything.)

SmartMedia and xD flash cards are notionally controller-less in the sense that the electrical interface is just parallel interface of NAND flash chip and all the FTL stuff is done by whatever device it's plugged into. one problem with that is that it requires all the devices to use same FTL logic with the standard one being NDA'd.

There ought to be some way to make a secure data transfer interface.

I guess not making it an universal bus is one of the steps for that. Also, making it either optical or magnetic should help.

facepalm

Two Words.

Legacy Hardware.