Marvel at the feat of operations, not the engineering. The techniques used to process this information over the years have mostly been quite banal. Look behind the scenes and much of it is just batch processes written in old school languages (Perl, Java, even C) reading text files from one folder, filtering or enhancing them, and writing them out to another. I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2).

> I once taught a PM how to use a Perl hash to do a join instead of nested for loops and she spent the next year feeling godlike because she could do this stuff in O(n) instead of O(n^2).

Loved this.

Correct I think that would be a better way to put it. None of the individual building blocks seem very technically challenging, but to consistently enforce it at a company wide scale seems quite cool.

What’s a PM, prime minister?

Because of the context, I'd assume "Project Manager", basically a low-level manager of sorts.

At Amazon "Program Manager", in this case really was a TPM or "Technical Program Manager". TPMs are effectively serial project managers, often charged with shepherding complex projects that require the efforts of more than one business or development group. Dealing with the data produced by those projects sometimes fell on them since it was difficult enough to persuade the developers to write the primary functionality.

i havent come to a personal conclusion on if its even a feat of engineering. Im impressed by amazon’s handling of logistics, and I do think consumer data helps Amazon know when and where products are most likely to sell, perhaps even upsell customers or get customers to agree to a steeper markup, but beyond that, i don’t think actual predictive models of consumer behavior has really been achieved here. Ive never gotten a sense that amazon intuits my consumer habits and ive made a lot of purchases over the years. i dont think amazon has made a dime on storing timestamps for when i turned a page on a kindle or learned anything from the music i listened to. ive never bought a single recommended product. On a scale of 1-10, 10 being Delos incorporated and perfectly modeling user behavior—-accurately predicting how i’ll react before I even see the products they place in front of me—and 1 being Wall-E making blocks of trash, id say this data collection is a 2.5 at best.

I would personally define efficacy of predictive modes to be a scientific achievement rather than engineering, but that might just be a vocab discrepancy.

Do you think it’s possible to determine how good their modeling is externally? I would assume we’d need an extensive peek behind the curtains to come to a conclusion.