News for hackers. Stuff that matters.
Re:How F5Bot Slurps All of Reddit by mustacheemperor on Monday July 30, 2018 @04:53PM (#17647915)
Reddit doesn't even allow users to save more than 1000 posts, and worse does not visibly document this or provide any kind of warning that the limit has been exceeded. Anecdotally, I've read users say that revisiting the saved pages will still show an "unsave" button so the information is recorded somewhere. But once a user exceeds 1000 entries on their "saved" page, adding new ones will silently vaporize old ones.

https://www.reddit.com/r/help/comments/6nxqjm/maximum_of_100...

Re:How F5Bot Slurps All of Reddit by Deimorz on Monday July 30, 2018 @06:00PM (#17648515)
It's a bit weirder than that. It actually does save all the posts, but the "saved" page (like almost every other page on the site) will only show you 1000 items, so there's just no way to access all the older items once they've been "pushed off the end".

I posted some more information about it a while ago here: https://www.reddit.com/r/help/comments/7en0uu/my_saved_posts...

Re:How F5Bot Slurps All of Reddit by lucb1e on Monday July 30, 2018 @08:19PM (#17649495)
If you unsave a newer one, wouldn't the older one pop up again because it updates the index with limit 1000 again? If the data is there, it should find all posts and then truncate, in that order, if I'm understanding this correctly.
Re:How F5Bot Slurps All of Reddit by Deimorz on Monday July 30, 2018 @08:33PM (#17649574)
Nope, because the old one's already been pushed off the end of the list and it doesn't re-generate the list when you unsave, just removes the item from the list.

Imagine I have a list of max length 5, when I initially fill it up it looks like [5, 4, 3, 2, 1]. If I save one more thing, it adds 6 at the front, then truncates the list and removes the 1 from the end, so now you have [6, 5, 4, 3, 2]. At that point, if I unsave #3, it just removes it from the list so you'd have [6, 5, 4, 2]. #1 is still saved, but nothing happens to pull it back into the list.

Re:How F5Bot Slurps All of Reddit by ralfd on Tuesday July 31, 2018 @12:09AM (#17650479)
What if you unsave 2 items and save 1. Is then the list remade?
Re:How F5Bot Slurps All of Reddit by Deimorz on Tuesday July 31, 2018 @01:55AM (#17650860)
No, the list is never remade. The new item just gets inserted into the list, exactly the same as if it had never reached the limit in the first place.
Re:How F5Bot Slurps All of Reddit by rotred on Monday July 30, 2018 @11:05PM (#17650240)
why would they make the limit so low?
Re:How F5Bot Slurps All of Reddit by Deimorz on Tuesday July 31, 2018 @01:58AM (#17650873)
Not sure, the 1000 limit is in the file from its very first version 10 years ago: https://github.com/reddit-archive/reddit/commit/33fd4e9684ca...

1000 probably seemed like a lot at the time or had reasonable performance, and it's just never been changed.

Re:How F5Bot Slurps All of Reddit by Puer on Monday July 30, 2018 @04:59PM (#17647976)
Saving posts is also notoriously unreliable, in my experience. I've had posts where I've clicked the "save" button and it's flipped to "unsave" but upon refreshing the page the button resets as if it was never pressed (and it never gets added.)
Re:How F5Bot Slurps All of Reddit by bluetidepro on Tuesday July 31, 2018 @08:27AM (#17652579)
A good work around tip is to set up an https://ifttt.com/ to trigger when you save to offload your saves to somewhere else. I save all my reddit saves to Evernote, personally. It can help with this known issue!