News for hackers. Stuff that matters.
Re:twitter/the-algorithm by influx on Monday April 25, 2022 @05:13PM (#31160689)
There's not an algorithm to order timelines?
Re:twitter/the-algorithm by jonwinstanley on Monday April 25, 2022 @05:20PM (#31160778)
If it’s a list of tweets ordered based on a kajillion ML data points that varies per user is it still an algorithm?

And does every user have their own algorithm?

And could it be made readable to a human?

Re:twitter/the-algorithm by abduhl on Monday April 25, 2022 @05:22PM (#31160799)
Yes, irrelevant, and the code somewhere is readable.

Take an example like gravity: is there an algorithm for how gravity acts even though it varies per each molecule based on a kajillion other molecules? Of course there is.

Re:twitter/the-algorithm by marcinzm on Monday April 25, 2022 @05:46PM (#31161117)
If I have your DNA can I recreate a perfect copy of you as you are now including your mind? It is after all the code you were made with.
Re:twitter/the-algorithm by vasco on Monday April 25, 2022 @05:59PM (#31161278)
The point is not to recreate it or explain it, or to have a way to say "tweet X will get rank Y" but to open source what there is. It doesn't matter if no single person can explain it, it'll still be open source and readable and the types of inputs will be known. But this doesn't look like a discussion on open source and more about politics at this point.

I for one haven't worked at FAANG and think it'll be super interesting to read this code, I can't believe software engineers are complaining about a potentially super complex bleeding edge codebase for timeline recommendations / ranking being open sourced. This is going to be great reading material if it ever gets published!

Re:twitter/the-algorithm by abduhl on Monday April 25, 2022 @06:08PM (#31161387)
Nobody is trying to “recreate a perfect copy” of Twitter. Do you think that code which produces random numbers is also not an algorithm because it doesn’t recreate a perfect copy of a previous random number?
Re:twitter/the-algorithm by stjohnswarts on Monday April 25, 2022 @06:57PM (#31161899)
No you can't, you can only make a twin, but the mind is shaped by experience also and not just genetics. As is the body.
[dead]
Re:twitter/the-algorithm by zbentley on Monday April 25, 2022 @05:56PM (#31161235)
Computer systems, even AI/ML-driven ones, aren't uncomprehended the way some areas of science that we haven't developed a strong understanding of are.

Systems comprised of (or generated by) comprehensible parts can be comprehended, we just often opt not to put in that work. There's a reason model interpretability is such a significant and increasingly-demanded area of expertise.

Re:twitter/the-algorithm by ironmagma on Monday April 25, 2022 @06:04PM (#31161344)
In that case, the definition of “the algorithm” is the training set.
Re:twitter/the-algorithm by JumpCrisscross on Monday April 25, 2022 @06:18PM (#31161487)
> the definition of “the algorithm” is the training set

Or the trained model itself. There are people looking for intentional bias. But the insidiousness of the problem likely arises from unintentional bias. Letting researchers brute force the ranking models with hypotheticals could be a win win.

Re:twitter/the-algorithm by ironmagma on Monday April 25, 2022 @07:37PM (#31162257)
Is there any reason to not think that like most human-created systems it has a mixture of intentional and unintentional bias?
Re:twitter/the-algorithm by lamontcg on Monday April 25, 2022 @06:19PM (#31161505)
Twitter itself is probably the training set for twitter.
Re:twitter/the-algorithm by ironmagma on Monday April 25, 2022 @07:35PM (#31162249)
In that case, the algorithm is the labels.
Re:twitter/the-algorithm by dmonitor on Monday April 25, 2022 @05:15PM (#31160715)
Turns out, it's been RNG this entire time.
Re:twitter/the-algorithm by noobermin on Monday April 25, 2022 @05:42PM (#31161055)
If that's the case, let it be client side RNG and not an RNG run server side, thanks.
Re:twitter/the-algorithm by legowo on Monday April 25, 2022 @11:55PM (#31163779)
what ups
Re:twitter/the-algorithm by Hamuko on Monday April 25, 2022 @05:21PM (#31160790)

  ORDER BY timestamp DESC;
Re:twitter/the-algorithm by dylan604 on Monday April 25, 2022 @05:27PM (#31160878)
no self respecting PM would ever allow something so trite and simple. this was obviously copy&pasted from stackoverflow. something like this would never web scale, so obviously they need somethign written from scratch preferably in Go/Rust/Node/anythingButSQL
Re:twitter/the-algorithm by talideon on Monday April 25, 2022 @05:24PM (#31160835)
The one true algorithm!
Re:twitter/the-algorithm by krapp on Monday April 25, 2022 @05:21PM (#31160788)
There probably is not "an algorithm" on a site as large and complex as Twitter, no. There are probably dozens if not hundreds of algorithms spread throughout the codebase which affect the timeline for individual users, possibly even code entirely self-generated by ML systems.
Re:twitter/the-algorithm by captainmuon on Monday April 25, 2022 @05:28PM (#31160891)
Well sure there is "ORDER BY" something, and there are batch jobs to calculate this something from different variables. Maybe it is partially a ML black box, but I'm pretty sure they are not flying completely blind. There's got to be an internal Wiki or some PPTs where they list what goes into the mix. Number of likes, retweets. Attached media, hashtags, trending or not. Do they do sentiment analysis and use positivity as a factor? Do they do PageRank so that likes from important people have more weight? Are there manual debuffs for certain topics? Are certain posts removed or added in a second pass? These are all answerable questions in principle.
Re:twitter/the-algorithm by jvanderbot on Monday April 25, 2022 @05:41PM (#31161046)
Maybe not. It could all be event-driven, distributed, and just pulling off various prioritized queues.
Re:twitter/the-algorithm by stjohnswarts on Monday April 25, 2022 @06:58PM (#31161910)
that's still an algorithm
Re:twitter/the-algorithm by jvanderbot on Wednesday April 27, 2022 @04:41PM (#31185517)
We'll, it's ok if you feel that way given this is entirely subjective, but I disagree based on experience.

I disagree b/c: That's a system. An algorithm is designed, a system will emerge from pieces. An algorithm can be defined, a system's behavior has to be characterized post-hoc. You can characterize a system, but only as a black/gray box. An algorithm has invariants and stateful steps, a system has nearly infinite state and nearly zero meaningful invariants.

Re:twitter/the-algorithm by cellis on Monday April 25, 2022 @05:52PM (#31161194)
The trouble is, there could be literally millions of ORDER BY queries. Hell, each user could be given a fine-tuned cnn…
Re:twitter/the-algorithm by onlyrealcuzzo on Monday April 25, 2022 @05:24PM (#31160832)
In the end, it's still code that is being executed, which can be called "an algorithm". Algorithms are under no obligation to be simple or only include "one algorithm".
Re:twitter/the-algorithm by krapp on Monday April 25, 2022 @05:29PM (#31160902)
So then "open sourcing the algorithm" likely means opens sourcing the entire Twitter codebase?
Re:twitter/the-algorithm by baryphonic on Monday April 25, 2022 @05:42PM (#31161068)
At this point, what harm would that do to Twitter? The company's tech has been stagnant for years, and the only real features recently have been attempts to exploit Twitter's network effects to ape Clubhouse & Substack. (Seems successful for the former, kind of a failure for the latter.)
Re:twitter/the-algorithm by stjohnswarts on Monday April 25, 2022 @06:59PM (#31161923)
do we know? has any angy ex-twitter engineer revealed the secret sauce to the world?
Re:twitter/the-algorithm by dragonwriter on Monday April 25, 2022 @05:33PM (#31160951)
The aggregate of a set of systems that maps inputs and state to outputs and state updates is...

still a system that maps inputs and state to outputs and outputs and state updates.

A collection of algorithms is just a bigger algorithm.

Re:twitter/the-algorithm by apgwoz on Monday April 25, 2022 @05:42PM (#31161060)
Turns out they never actually scaled their mailbox pattern. As a result, what you see is what they delivered before timeout.
Re:twitter/the-algorithm by jdrc on Monday April 25, 2022 @05:22PM (#31160797)
just birds poking random tweets