1. Absolutely incredibly complex architecture. The recommendation engines are not just single libraries that can be easily open sourced or transformed to a plugin system.
2. Recommendation engines exist to push up relevant metrics. These are either clear wins for users (like reported satisfaction), mixed wins for users and businesses (content engagement), or clear wins for businesses (ad engagement). Most businesses aren't thrilled about subbing in systems that degrade metrics.
How does an algorithm "that attempts to prioritize nuanced conversations about important topics" work? (Or "to find mind-expanding threads", for "savage dunks" or "thirst traps of hot new snax"? -- other examples from the post.)
I suppose you spend some time with existing tweets and ML and develop a model that can produce a score of some sort on these concepts for a tweet, and then run every tweet through the model and present the high-scoring ones. Of course, you can't just look at individual tweet, which in isolation don't mean much, but also at how it fits into a conversation. (For that matter, I'd be interested to see what a model of a conversation is.)
Sounds expensive and quite possibly not accurate enough to be worthwhile.
It also seems like a major strategic decision to give third parties every tweet by everyone. It seems like the business changes from being what twitter is now to a message routing backend, and these third parties become what twitter used to be. That's a fundamental shift, that probably devalues the company by an order of magnitude, since they would be dissolving the valuable thing they have -- the social network they have.
Just doesn't make any sense to me.
edit: complexity lol
I don't think "marketplace of algorithms" necessarily mean people need to be able to make arbitrary database queries either, but seems besides the point.
Having their own DSL for sorting timelines can also solve that problem nicely and within whatever performance requirements they would end up with.
You mean Turing-complete?
Not sure why this is getting downvoted. It's not crazy. Even Jack is pushing for it: "The choice of which algorithm to use (or not) should be open to everyone"
https://twitter.com/jack/status/1507146276416098307