Re:Show HN: Export HN Favorites to a CSV File
by gabrielsroka on Monday April 06, 2020 @12:25PM
(#22795044)
In Python, `range(1, 17)` produces the numbers 1-16. I hard-coded it just for your favorites.
A better way to solve it is to look at the `len()` of the results, and stop when it gets to 0:
p = 1
while True:
r = requests.get(f'https://news.ycombinator.com/favorites?id=app4soft&p={p}')
s = BeautifulSoup(r.text, 'html.parser')
faves = [{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')]
print(faves)
p += 1
if len(faves) == 0:
break
Re:Show HN: Export HN Favorites to a CSV File
by app4soft on Monday April 06, 2020 @01:54PM
(#22796055)
Great!
Re:Show HN: Export HN Favorites to a CSV File
by gabrielsroka on Monday April 06, 2020 @11:53PM
(#22800222)
I think this one is a little cleaner. I used some of the ideas in sbr464's code.
path = 'favorites?id=app4soft'
while path:
r = requests.get('https://news.ycombinator.com/' + path)
s = BeautifulSoup(r.text, 'html.parser')
print([{'title': a.text, 'url': a['href']} for a in s.select('a.storylink')])
more = s.select_one('a.morelink')
path = more['href'] if more else None
One more question: what is the best way stop it when it will reaches last page?
> for p in range(1, 17):
Actually p=17[0] is empty (as p=16 is maximum as for now).
Maybe, script should scrap pages from `1` to `infinity` UNTIL it detect next message on page[0]:
> app4soft hasn't added any favorite submissions yet.
[0] https://news.ycombinator.com/favorites?id=app4soft&p=17