News for hackers. Stuff that matters.
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by Mkengine on Monday October 21, 2024 @11:49AM (#41905419)
Does it support http://fanfiction.net/ ? I never found an easy solution for that one.
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by maoserr on Monday October 21, 2024 @12:14PM (#41905667)
You can import a csv of all the chapter links, looks like it's just incremental numbering in the url
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by t-3 on Monday October 21, 2024 @02:15PM (#41906788)
The issue is most likely cloudflare blocking most the best scraping methods. If the site can be pulled down with eg. wget or curl without a bunch of options that you definitely aren't writing by hand, pandoc can just be used to directly make an epub.
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by pasc1878 on Monday October 21, 2024 @12:46PM (#41905946)
I use a calibre add-in https://www.mobileread.com/forums/showthread.php?t=259221

It sort of works ie some stories just work others just get the first page.

Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by seridescent on Monday October 21, 2024 @01:41PM (#41906438)
you can export epubs from https://fichub.net/
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by kemayo on Monday October 21, 2024 @04:30PM (#41908224)
Fanfiction.net is trivial... apart from it having Cloudflare bot blocking turned up to aggressive levels. I've not seen an approach that works, other than using headless browsers to fetch the content.
Re:Show HN: Epublifier – scrape pages (books, manuals) for o... by theultdev on Monday October 21, 2024 @06:45PM (#41909356)
headless browsers won't work by default for cloudflare captchas.

open source stealth plugins don't really work now either.

you have to use real browser fingerprints.