News for hackers. Stuff that matters.
Re:Show HN: Export HN Favorites to a CSV File by app4soft on Sunday April 05, 2020 @07:05PM (#22789488)
Could someone convert it to Python-script?
Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Sunday April 05, 2020 @08:49PM (#22790039)
Part of the advantage of running JavaScript in your browser is that you might already be authenticated and it can use your session. But, fetching your HN favorites doesn't require authentication.

  #!/usr/bin/env python3
  import requests
  from bs4 import BeautifulSoup

  for p in range(1, 17):
      r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
      s = BeautifulSoup(r.text, 'html.parser')
      print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])
Re:Show HN: Export HN Favorites to a CSV File by app4soft on Monday April 06, 2020 @11:32AM (#22794600)
Thanks!

One more question: what is the best way stop it when it will reaches last page?

> for p in range(1, 17):

Actually p=17[0] is empty (as p=16 is maximum as for now).

Maybe, script should scrap pages from `1` to `infinity` UNTIL it detect next message on page[0]:

> app4soft hasn't added any favorite submissions yet.

[0] https://news.ycombinator.com/favorites?id=app4soft&p=17

Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Monday April 06, 2020 @12:25PM (#22795044)
In Python, `range(1, 17)` produces the numbers 1-16. I hard-coded it just for your favorites.

A better way to solve it is to look at the `len()` of the results, and stop when it gets to 0:

  p = 1
  while True:
      r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
      s = BeautifulSoup(r.text, 'html.parser')
      faves = [{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]
      print(faves)
      p += 1
      if len(faves) == 0:
          break
Re:Show HN: Export HN Favorites to a CSV File by app4soft on Monday April 06, 2020 @01:54PM (#22796055)
Great!
Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Monday April 06, 2020 @11:53PM (#22800222)
I think this one is a little cleaner. I used some of the ideas in sbr464's code.

    path = 'favorites?id=app4soft'
    while path:
        r = requests.get('https://news.ycombinator.com/' + path)
        s = BeautifulSoup(r.text, 'html.parser')
        print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])
        more = s.select_one('a.morelink')
        path = more['href'] if more else None
Re:Show HN: Export HN Favorites to a CSV File by app4soft on Tuesday April 07, 2020 @09:21AM (#22802940)
Go on! ;)
Re:Show HN: Export HN Favorites to a CSV File by death-by-ppt on Sunday April 05, 2020 @09:43PM (#22790244)
Can someone convert it to Bel?
Re:Show HN: Export HN Favorites to a CSV File by amjd on Monday April 06, 2020 @12:46AM (#22790964)
I had written a Python script to get saved (upvoted) links as JSON / CSV a few years ago: https://github.com/amjd/HN-Saved-Links-Export

I'm not sure if it still works as it too relied on HTML scraping. Perhaps I should update it to support favorites too.

Edit: Whoa, it's been 4 years already. I believe HN didn't have favorite feature at the time. That's why I used upvoting as my bookmarks system and created a script to export that data.

Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Monday April 06, 2020 @02:18AM (#22791281)
@amjd, thanks for sharing. I upgraded it from Python 2 to Python 3, but when I ran it, I got a 404 error on the `saved` endpoint. Does it work for you?

Edit: I see from the other PR it's called `upvoted`.

Edit 2: I changed it to `upvoted` and now I get a 200 OK, but the code crashed right afterwards on `tree.cssselect()`.

Re:Show HN: Export HN Favorites to a CSV File by amjd on Monday April 06, 2020 @06:09AM (#22792253)
It's four years old so the page HTML structure must have likely changed. I'll take a look at it when I'm free and update the script. Thanks for your PR, I'll review and merge the two open PRs as well.

Btw, good work on your JS solution. It's great that it just works without requiring any download or installation. :)