Re:Report: WeWork expected to cut 500 tech roles
by nostrademons on Tuesday October 08, 2019 @11:33PM
(#21199755)
The percentage of changes that actually improve things drops significantly as the company gets more optimized, so you need to try more things to have a good chance of actually getting one implemented.
Say that a tweak to the page improves ad revenue by $50M. 98% of tweaks have neutral or negative effects. The expected value of the tweak is thus $1M, well over one engineer's salary. You need to have 50 of them going on in parallel to have a good chance of actually finding a successful tweak, though. (Well, technically about 150 to have a 95% confidence that at least one will prove useful, but there's also a good chance that more than one will be useful then.) That's what all the engineers are doing - independently implementing different changes and discarding 98% of the results.
When I did this kind of work, the vast majority of engineering time was actually spent measuring the results. It was not unusual for the actual implementation work to be done in about 3 hours, but the experiment design, approval, configuration, logging, data-collection, and analysis to take 3 weeks. And that was with a fair amount of existing tooling for measurement & process.
Say that a tweak to the page improves ad revenue by $50M. 98% of tweaks have neutral or negative effects. The expected value of the tweak is thus $1M, well over one engineer's salary. You need to have 50 of them going on in parallel to have a good chance of actually finding a successful tweak, though. (Well, technically about 150 to have a 95% confidence that at least one will prove useful, but there's also a good chance that more than one will be useful then.) That's what all the engineers are doing - independently implementing different changes and discarding 98% of the results.
When I did this kind of work, the vast majority of engineering time was actually spent measuring the results. It was not unusual for the actual implementation work to be done in about 3 hours, but the experiment design, approval, configuration, logging, data-collection, and analysis to take 3 weeks. And that was with a fair amount of existing tooling for measurement & process.