News for hackers. Stuff that matters.
Re:Tech compensation in 2021 by hintymad on Thursday October 14, 2021 @11:54PM (#28874066)
> My point was not that algorithms are irrelevant but they are table stakes

You're wrong. A previous team built a simple data processing pipeline for a big customer. The throughput is single digit per second. The customer was furious as they would have wait for days to get their data in. The reason is the team was not aware that their graph building and traversal algorithms are O(n^2), and they didn't understand how to distribute their processing logic. My team came in, fixed the algorithms with merely dozens of lines, and improved the throughput by more than 200 times in a matter of days. If you build large backend for data intensive systems, which pretty much every tech company needs, you will need algorithms and data structures. Probabilistic data structures? Check. Manually optimizing popular machine learning algorithms? Check. Rolling out our own implementation of graphs, trees, vector similarity search algorithms and indices, parallelization algorithms, cache oblivious data structures? Check, check, and check. You don't have to use those techniques, but your value to your projects will increase exponentially if you push the boundary.

Re:Tech compensation in 2021 by throwaway2016a on Friday October 15, 2021 @12:06AM (#28874163)
What part of my statement is wrong?

I said algorithms are table stakes (a poker term for something you need to know just to play at the table). To continue the analogy, sounds like whoever designed that system didn't have enough to buy into the table.

> If you build large backend for data intensive systems

I have. Many.

Re:Tech compensation in 2021 by hintymad on Friday October 15, 2021 @12:22AM (#28874260)
ouch, my bad. Somehow I read “algorithm is irrelevant”, and you’re right, I didn’t get your analogy.
Re:Tech compensation in 2021 by Aeolun on Friday October 15, 2021 @05:14AM (#28875815)
200x single digit/second still seems lowish? I’m not sure if I’d call that a large backend for a data intensive system.

Depends on how extensive your processing is of course.