News for hackers. Stuff that matters.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by joewrong on Thursday November 21, 2019 @05:21PM (#21600375)
" ORDER BY tweet.time DESC "

sign me up

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mc3 on Thursday November 21, 2019 @05:33PM (#21600470)
Whether it is the lack of an ORM or lack of a "algorithm" that pleases you, this comment has made my day :-).
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by hombre_fatal on Thursday November 21, 2019 @07:26PM (#21601271)
I believe they're referring to the lack of algorithmic sorting.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by Jaxan on Friday November 22, 2019 @09:04AM (#21605609)
I’m not even sure what to think of the term “algorithmic sorting.” Is quicksort considered as algorithmic sorting ... ?
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by hombre_fatal on Friday November 22, 2019 @02:52PM (#21609368)
I'm referring to "algorithmic" in its negative usage as short-hand for what you can see as of late with regard to Twitter, Youtube, and Facebook's suggestion engines where more complex sorting and filtering is happening than a simple ORDER_BY.

Sure, not the best short-hand, but it seems most people understood what I meant.

For example, https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Like this submission: https://news.ycombinator.com/item?id=20184282 where people will shorten the "evil" to just "the algorithm".

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by redwall_hp on Friday November 22, 2019 @11:09AM (#21606851)
That SQL statement is literally algorithmic sorting...
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mc3 on Sunday November 24, 2019 @03:44PM (#21623242)
Yes, I hate it when they manipulate the masses like that. They're so time-ist! Show me that old stuff!
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by npo9 on Thursday November 21, 2019 @06:07PM (#21600752)
It’s the pure simplicity.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by anonytrary on Thursday November 21, 2019 @11:00PM (#21602302)
Pure simplicity works well when you have little to no users. It's reasonable to think that

  ORDER BY tweet.time DESC
would result in a impractical experience for a site that has millions of DAU.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by lvturner on Friday November 22, 2019 @12:59AM (#21603172)
Only if you subscribe to everything - if you keep your follower list small, the individual experience doesn't change regardless of the overall size of the platform.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by Nextgrid on Friday November 22, 2019 @08:04AM (#21605166)
Why would it be? It'll naturally keep your list of followings small (as a large one will be unmanageable) and promote quality over quantity.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by dblohm7 on Friday November 22, 2019 @01:57AM (#21603433)
Personally, I’m pleased by both!
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by jamroom on Thursday November 21, 2019 @05:53PM (#21600642)
I'm not an PG user but there does not appear to be an index on tweet.time - and even if there was wouldn't it better to do ORDER BY tweet.id DESC? I assume ordering by primary key is going to be the fastest, and would (ideally) prevent maintaining an index on tweet.time.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by chaps on Thursday November 21, 2019 @05:58PM (#21600678)
Sure, but you're making some assumptions in doing so. For example, that the time and id columns will remain constant. It might be a decent assumption in the beginning, but once you start doing updates on the table, all bets are off.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by WorldMaker on Thursday November 21, 2019 @06:58PM (#21601124)
Yup, as soon as you need to change ID schemes, you risk breaking query logic. Sure things like v1 UUIDs and Snowflakes (and ULIDs and so forth) try to maintain temporal ordering in their ID formats, but what if you need v4 UUIDs for better clustering in some hash-table or SHA-256 hashes for some sort of content addressing scheme?

Also, it's just a very premature optimization in a world where CLUSTERED INDEXes exist. The database engine doesn't have to cluster by primary key, it can cluster directly by time (or any other index) if you ask it to. The power of doing it that way is that you can flexibly change it based on real performance issues (how do your execution plans look?) and characteristics (are you read heavy or write heavy? which ones are your bottle necks?), whereas if your application makes assumptions about ID format it's a lot harder to on-the-fly tune queries that all need to be rewritten.

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by ErikAugust on Thursday November 21, 2019 @06:52PM (#21601085)
I am not sure IDs are exactly incremental in Twitter at this point. I think they are issued by blocks or something of that nature. Someone here could answer that far better.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by WorldMaker on Thursday November 21, 2019 @07:00PM (#21601135)
Twitter calls them "snowflakes". It's a 64-bit ID of that consists of a tuple of sorts of a timestamp, machine code, sequence ID. They start with timestamps to mostly give them monoticity in the direction of time (ie, sort order them), but they definitely aren't simple sequence IDs.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by axaxs on Thursday November 21, 2019 @06:10PM (#21600777)
Assuming that every tweet.time is unique, what benefit would you gain from indexing it?
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mc3 on Thursday November 21, 2019 @09:43PM (#21601969)
With the query given, the optimiser can immediately figure out to get the latest record from the index and scan back through the index. If the index has included columns, it could scan the data straight off the index. Without the index you need to scan the entire table, sort it in memory, and then read off the top columns. If you were doing a top X query it would be more markedly faster by fetching less data from disk. But I think that query is getting all the records, but still it will be quicker by avoiding the in memory sort.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by axaxs on Thursday November 21, 2019 @11:09PM (#21602328)
interesting, thanks for the insight. I haven't touched DB setup in many years, and even then was novice. The best person I knew told me to index if a column wasn't unique, but also wasn't something with only two or three choices. Sounds like I have more reading to do...
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mc3 on Sunday November 24, 2019 @03:42PM (#21623218)
I'm certainly not an expert - I did a great DB course about 15 years ago and then used the skills every now and then since. I might not be up on the latest. And I am more of a SQL Server person. BUT... the main thing I learn is view actual execution plans, and see what is actually happening before adding indexes (unless it's an 'obvious' index).

> The best person I knew told me to index if a column wasn't unique, but also wasn't something with only two or three choices.

Yeah I think this is too broad advice, and you need to understand what you want to achieve. Mentally, choosing indexes is like choosing whether to use a hashtable vs. for loop vs. binary tree etc in an algorithm in code. There is not golden rule or "always use a hashtable if there are 100 entries" type thing. You just need to figure it out on a case by case basis. And usually there are only 2-3 tables in your DB worth a lot of effort in figuring it out!

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by vntok on Thursday November 21, 2019 @06:26PM (#21600886)
Why would it be unique?
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by axaxs on Thursday November 21, 2019 @06:37PM (#21600951)
So, in fairness I didn't dig through the code to see the column, but assumed some granularity like milliseconds? It was a genuine question on whether it would speed up the query at all if indexed since it is generally grabbing everything then sorting.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by spullara on Thursday November 21, 2019 @09:37PM (#21601947)
Indexes generally give you sorting for free so you don't have to look at everything.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by gigatexal on Thursday November 21, 2019 @07:04PM (#21601158)
In general if it is something you will query in a where clause it should be indexed.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by WA on Friday November 22, 2019 @08:27AM (#21605335)
Just create a private list on Twitter, put all people you follow in there. Bam, done. Reverse-chronological order, no ads.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by nkozyra on Thursday November 21, 2019 @05:34PM (#21600479)
I'll never understand why the stream cannot just default to logic like this or [my follows]+[non-rt/replies]+[desc by time]

Best I can come up with is it'd make promoted comment more apparent?

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by hombre_fatal on Thursday November 21, 2019 @07:32PM (#21601320)
There are good faith arguments for applying smarter sorting / filtering to the stream.

For example, if someone follows two people and one person simply tweets at 1000x the rate of the other, I would argue that it would be better UX if the rare tweet from the other person gets more weight.

Another thing Twitter does is show you popular things that have been liked by the people you follow. It might be way too much to show you every thing that every person you follow has liked every time. But it can introduce some interesting content to your stream to see such second-tier content.

I don't see how promoted content plays into it since you can interpolate it anywhere.

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by MagnumPIG on Thursday November 21, 2019 @05:41PM (#21600529)
That's exactly the reason. Ads can much more easily be camouflaged as real content this way.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by WilTimSon on Thursday November 21, 2019 @06:43PM (#21600995)
If only they weren't inane. I think Twitter is probably the worst in terms of 'promoted content' because it always seems to serve up the same ad several days in a row and it's always something surprisingly off-topic. I'd be browsing my feed full of political news, movie trailers, and animal photos... and get an ad for ketchup. How on Earth could I confuse that for a real tweet or something my friends would retweet? A ketchup ad? Mental.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by hombre_fatal on Thursday November 21, 2019 @07:39PM (#21601358)
This doesn't make much sense to me. You don't need complicated algorithmic sorting and filtering to inject an ad space after every fifth tweet. Even the client can do that.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by megous on Thursday November 21, 2019 @08:32PM (#21601666)
Client does that. If you consume twitter via their API, you'll notice there are no ads there.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by hirundo on Thursday November 21, 2019 @06:15PM (#21600808)
You see many more outlier tweets with thousands of likes when your feed is interspersed with "most liked by follows and follows of follows", as opposed to just "tweeted by follows". I think that improves the signal to noise ratio of the feed.

But yeah, I wish a simple timeline was the default, and twitter stopped switching away from it.

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mipmap04 on Thursday November 21, 2019 @09:06PM (#21601827)
For anyone interested in this, google “hacker news gravity algorithm” to see an interesting example of how sorting was done here at one time (as I understand it the algorithm has evolved).
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by mipmap04 on Thursday November 21, 2019 @09:03PM (#21601814)
I was working on creating an rss reader with a reddit-like interface that allowed follows, voting, comments, etc. The computational expense of a fancy sort algorithm based on factors other than just one non-computed column is immense once you get to a decent number of users. I ended up removing the algo just to lower my hosting bills. Of course, there were other potential solutions, but for a side project, this was easiest.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by kgthegreat on Friday November 22, 2019 @05:56AM (#21604505)
Feel free to take this for a spin - http://trysensible.com/

It has that and more!

Open source - https://github.com/kgthegreat/sensible

Re:Simple-twitter: A bare-bones Twitter clone implemented in... by robobro on Thursday November 21, 2019 @09:50PM (#21602007)
Pleroma, part of the fediverse (3,500,000+ users) has exactly this.
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by singron on Friday November 22, 2019 @12:07AM (#21602817)
Is "Latest" sorting in the twitter UI not this?
Re:Simple-twitter: A bare-bones Twitter clone implemented in... by sam_lowry_ on Thursday November 21, 2019 @05:48PM (#21600596)
Fenix seems to have that.