Show HN: Export HN Favorites to a CSV File

Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Sunday April 05, 2020 @08:49PM (#22790039)

Part of the advantage of running JavaScript in your browser is that you might already be authenticated and it can use your session. But, fetching your HN favorites doesn't require authentication.

  #!/usr/bin/env python3
  import requests
  from bs4 import BeautifulSoup

  for p in range(1, 17):
      r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
      s = BeautifulSoup(r.text, 'html.parser')
      print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by app4soft on Monday April 06, 2020 @11:32AM (#22794600)

Thanks!

One more question: what is the best way stop it when it will reaches last page?

> for p in range(1, 17):

Actually p=17[0] is empty (as p=16 is maximum as for now).

Maybe, script should scrap pages from `1` to `infinity` UNTIL it detect next message on page[0]:

> app4soft hasn't added any favorite submissions yet.

[0] https://news.ycombinator.com/favorites?id=app4soft&p=17

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Monday April 06, 2020 @12:25PM (#22795044)

In Python, `range(1, 17)` produces the numbers 1-16. I hard-coded it just for your favorites.

A better way to solve it is to look at the `len()` of the results, and stop when it gets to 0:

  p = 1
  while True:
      r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
      s = BeautifulSoup(r.text, 'html.parser')
      faves = [{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]
      print(faves)
      p += 1
      if len(faves) == 0:
          break

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by app4soft on Monday April 06, 2020 @01:54PM (#22796055)

Great!

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by gabrielsroka on Monday April 06, 2020 @11:53PM (#22800222)

I think this one is a little cleaner. I used some of the ideas in sbr464's code.

    path = 'favorites?id=app4soft'
    while path:
        r = requests.get('https://news.ycombinator.com/' + path)
        s = BeautifulSoup(r.text, 'html.parser')
        print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])
        more = s.select_one('a.morelink')
        path = more['href'] if more else None

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by app4soft on Tuesday April 07, 2020 @09:21AM (#22802940)

Go on! ;)

[ Parent | on HN ]

Re:Show HN: Export HN Favorites to a CSV File by death-by-ppt on Sunday April 05, 2020 @09:43PM (#22790244)

Can someone convert it to Bel?

[ Parent | on HN ]