Did you ask him why he took the approach he took?
Yes, and why random data? Possibly he needed 'real' data because the store used compression/RLE.
But certainly they should have got an estimate first - given a probability and size.
He was not using real data because we anticipated that with this new feature, the usage would change. (Making our existing data meaningless).
And there was no compression or anything fancy.
Re:Ask HN: Trapped by knowledge
by foobar_yc00 on Sunday January 01, 2017 @08:15PM
(#13298434)
I think he is being realistic and i think he did a good job, you wouldn't write this if you asked him or if you knew things like worst-case, average-case, and best-case performance.
I'm sorry but I can't agree with that unless you provide an explanation of why you need to measure instead of compute.
If you generate N data (N being choosen randomly in a uniform distribution between 100 and 200), and your data size is S (taken randomly between 50 and 500)..
The best, average and worst case scenario are easy enough to compute, aren't they ? Of course there was more than two variables (maybe 5), but it was not that hard.
One day he was asked to estimate the memory consumption of a new project. He did it by writing a simulator that would fill a data structure with random data. Each data size had a specific probability to be generated. Then he would measure memory used.
Why didn't he simply compute the total size ? That's basic math/CS.
There are stupid people and smart people. Being expert doesn't mean they are smart.