News for hackers. Stuff that matters.
Re:twitter/the-algorithm by shrimpx on Monday April 25, 2022 @05:39PM (#31161015)
I don't understand the concept of open-sourcing "the algorithm".

First of all, "the algorithm" is probably hundreds of thousands of lines of code, including all the tedious boilerplate like cache policies and multi-AZ logic.

And second of all, doesn't the algorithm include machine learning components, which are trained on terabytes of data? That data will likely be impossible to open source. And open sourcing the neural nets without the training data is mostly meaningless from a transparency perspective?

Re:twitter/the-algorithm by owaislone on Monday April 25, 2022 @05:55PM (#31161225)
Open sourcing is in this is not about the implementation or the CS algorithm. It is more about transparency. I think the idea is that the public should know how tweets are ranked, why tweets show up in timelines and which timelines, what makes tweets popular etc. Imagine Google publishing a document detailing how their system ranks pages aka publishes internal SEO rules officially. I don't know if it is a good idea or not. People with enough resources might be able to game the system (if they don't do it already).
Re:twitter/the-algorithm by Traster on Monday April 25, 2022 @06:22PM (#31161533)
The only change is that people with resource would stop guessing how to game the system, and start employing people to ensure they systematically game the system.
Re:twitter/the-algorithm by rootusrootus on Monday April 25, 2022 @07:00PM (#31161943)
> if they don't do it already

That's an interesting point. A practical description of the algorithm from the perspective of someone trying to game it may be more useful than anything Twitter or Google would release.

Re:twitter/the-algorithm by WORMS_EAT_WORMS on Monday April 25, 2022 @05:44PM (#31161085)
Probably right but I think trying to be transparent is better than not trying at all.

Gesture carries weight to the users too.

Not sure any big company has tried this before. I could be wrong, but either way looking forward to it / FWIW hope it catches on.

Re:twitter/the-algorithm by ProAm on Monday April 25, 2022 @05:43PM (#31161074)
Or is it six cats in a basement with a laser pointer and a mouse?

The point of releasing it is to let people know exactly why they see the tweets they do in the order they do. I hope Elon just goes back to time base ordering of tweets.

Re:twitter/the-algorithm by LordDragonfang on Monday April 25, 2022 @05:49PM (#31161146)
I personally know a FAANG employee whose full time job is building tools to try and help understand why the company's recommendation algorithm picks the things it does (and more specifically, predict how changes to the algo will affect that).

Even the people who build these systems barely know what the algorithm is going to do, much less why. It will be a herculean task to try and convey that to an average user.

Re:twitter/the-algorithm by shrimpx on Monday April 25, 2022 @07:24PM (#31162162)
Right. Also, it takes a new eng hire N years to gain a working understanding of the small subset of code that their team produces.
Re:twitter/the-algorithm by nthot on Monday April 25, 2022 @05:49PM (#31161143)
Disclaimer: I'm pretty new to twitter, so I may be misunderstanding something. On my Twitter Home Screen there are three little stars on the upper right of the central container which allow you to toggle between "top tweets" (ie. 'The Algorithm') and "latest tweets" which is the time based ordering.
Re:twitter/the-algorithm by UncleMeat on Monday April 25, 2022 @05:57PM (#31161252)
What you are misunderstanding is that there is more social clout to be gained by complaining about twitter than exploring the available features. The ability to sort by latest doesn't stop people from feeling self righteous by demanding this feature exist.
Re:twitter/the-algorithm by stjohnswarts on Monday April 25, 2022 @07:01PM (#31161953)
You understand it, it's just like instagram, facebook, reddit etc, you let them pick something for you via some algo or you order by date.
Re:twitter/the-algorithm by buu700 on Monday April 25, 2022 @06:03PM (#31161325)
Maybe it's a room full of lava lamps.
Re:twitter/the-algorithm by jonas21 on Monday April 25, 2022 @05:59PM (#31161273)
This is pretty standard in the machine learning world. You'll open-source the code and weights trained on a public data set (these are often licensed specifically for non-commercial use). But in production, you'll be using different weights trained on a proprietary data set.
Re:twitter/the-algorithm by vijaybritto on Monday April 25, 2022 @05:45PM (#31161104)
The whole point of the repo seems to just show that there's nothing called as "the algorithm" It's probably something like 100s of 1000s of algorithms doing different things
Re:twitter/the-algorithm by threeseed on Monday April 25, 2022 @05:43PM (#31161071)
You can open source the model code.

And developers will be able to train a model using it on a subset of Twitter data. Just that the quality of the outcome won't be the same as having the full set of Twitter data.

Re:twitter/the-algorithm by thorgutierrez on Monday April 25, 2022 @06:18PM (#31161493)
If it's too complicated, there is a good chance Elon will ask to simplify it until it can be open sourced.
Re:twitter/the-algorithm by mrkramer on Monday April 25, 2022 @05:43PM (#31161072)
Ever heard of Pseudocode which happens to be human-readable? If they are really going do it they will release source code for programmers and in general computer scientists to analyse and on top of that they will release pseudocode which non-techincal people can somewhat understand.