Re:What you need may be “pipeline +Unix commands” only
by JoeSmithson on Sunday March 03, 2019 @06:41AM
(#19293997)
> how often does that come up?
It's an important point to remember that a lot of things involved in human society have not exploded in size or complexity in the last 30 years.
Many data sets are basically proportional to the human population (health records, criminal records, property records etc), and these have been measured in the millions for 30+ years. In the same time the compute power of a single script has moved from the millions into the billions.
It's important, because if a government needs to, say, calculate something involving "every building in the country", or "everybody with a criminal record" they need to understand that this task, in 2019, can in fact be done by a single programmer parsing flat text files on their MBP, and does not need a new department.
It's an important point to remember that a lot of things involved in human society have not exploded in size or complexity in the last 30 years.
Many data sets are basically proportional to the human population (health records, criminal records, property records etc), and these have been measured in the millions for 30+ years. In the same time the compute power of a single script has moved from the millions into the billions.
It's important, because if a government needs to, say, calculate something involving "every building in the country", or "everybody with a criminal record" they need to understand that this task, in 2019, can in fact be done by a single programmer parsing flat text files on their MBP, and does not need a new department.
This is a bit like Grace Hopper always pointing out the difference between a microsecond and a nanosecond - https://www.youtube.com/watch?v=JEpsKnWZrJ8