News for hackers. Stuff that matters.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by jlink on Friday February 24, 2017 @07:53PM (#13729343)
who would be interested by an online website doing the job?
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by logicallee on Friday February 24, 2017 @10:29PM (#13729868)
if you really want to rake it in, serve, at static speeds (meaning instantly, I swear, boot a ramdrive (Tmpfs) and serve static html from nginx all from RAM), text versions of the top 10,000 web sites. there is so much crap on most sites. re-crawl hourly.

monetize via Google adwords.

EDIT: I'm not sure why I'm being downvoted. I am not suggesting serving PDF's. I am suggesting serving tiny text renders of top sites, that otherwise are much too bloated.

the hard part is getting the text and layout right. many people read many sites for the text IMO.

So I am suggesting you make an all-text version.

As an example, the front page of the New York Times right now, copied into Microsoft Word, is 2504 words. When I save from the word I copied into into .txt - I get a 16.4 KB file.

By comparison, when I put the site into a Page Size Checker -- http://smallseotools.com/website-page-size-checker/ -- I get 214.23 KB. That is impressively small, and it's a fast page.

If I try their competition, the Washington Post, I get 237 KB. If I try the Wall Street Journal, I get 938.15 KB -- nearly a full Megabyte. (This is actually more what I was expecting - I'm impressed by the Times.)

Suppose someone desperately wants to glance at the Wall Street Journal from a poor connection where they barely get data. The difference between 12 KB and nearly a megabyte is huge. Its the difference between 4 seconds and 312 seconds: 4 seconds as compared with 5 full minutes.

So there is a large need in my opinion for such a service in case someone desperately wants to see a text render. Preserving any formatting at all, helps hugely.

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by cozzyd on Friday February 24, 2017 @11:10PM (#13729990)
You can use the SHA-1 of the PDF's to avoid serving the same pdf twice.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by coulix on Saturday February 25, 2017 @01:53AM (#13730447)
SHA-256 ;)
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by vgb2k11 on Saturday February 25, 2017 @05:19AM (#13730937)
You've been away from HN a few days? The SHA-1 collision example uses PDFs in its demo. Hence other commenter saying SHA 256
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by jhlgkhkhil on Saturday February 25, 2017 @09:22AM (#13731687)
The OP was almost certainly being sarcastic.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by logicallee on Saturday February 25, 2017 @09:53AM (#13731787)
For clarity can you edit your comment to add cozzyd (the OP you mention) - I am sometimes sarcastic but not in this case. I'll then delete this comment.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by yeldarb on Saturday February 25, 2017 @02:54AM (#13730568)
Sounds cool and all but two huge problems:

1) Copyright: completely re-serving the complete content of the top 100 sites with your own ads does not fall under fair use and would almost certainly be a magnet for lawsuits.

2) Distribution: how do you find your niche of people with poor internet connections and get them to use your mirror instead of whatever site it is they want to read?

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by logicallee on Saturday February 25, 2017 @06:33AM (#13731121)
no clue on 2. for 1, you could have it be "opera mini/turbo as a service" so that you are arguing you are just shifting the viewer to the site, but it's still the user doing the viewing. it helps if you preserve any text ads on the site (or links, with alt-text, given you're probably not doing images. you could also replace images with a grainy black-and-white very low-fidelity version, this also shifts most ads on the original site, without adding hugely to your footprint.) To be honest I also thought perhaps javascript etc could be run, so that the heaviest sites of all are still downloaded and then turned into text versions. In many cases that can let someone browse a site that is otherwise incredibly slow.

This isn't legal advice, just the approach I would use off of the top of my head. I agree with you that it's hard. with the framework "opera minifier/turbofier as a service" it could work, though. Like a remote browser. (in a VM). Like, present it as "lynx as a service." (Lynx being an old terminal-based text browser.) Something like that, anyway.

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by fsiefken on Saturday February 25, 2017 @03:57AM (#13730715)
Doesn't Opera Mini or Turbo already provide this sevice? Perhaps add PPMD proxy text compression with an English dictionary with a JavaScript browser plugin on top of that. You can't get more efficient than that
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by logicallee on Saturday February 25, 2017 @06:29AM (#13731107)
Maybe, but asking someone to use a new browser is asking a lot. If you like, you can think of this as Opera minifier/turbofier as a service.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by rodw on Friday February 24, 2017 @09:15PM (#13729592)
For what it's worth, here's a service that does that https://documentalchemy.com/demo/pdf2txt (and more: https://documentalchemy.com/demo)
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by flexie on Saturday February 25, 2017 @10:04AM (#13731829)
Just tried the demos on this website.

I tried to extract text from a pdf that already has searchable text, which can be copy-pasted. This should be the easiest task of all but it made mistakes in every second word.

Then I asked the website to make a pdf into a word-file. It just inserted the whole pdf as a picture in word.

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by rodw on Saturday February 25, 2017 @11:37AM (#13732281)
> Then I asked the website to make a pdf into a word-file. It just inserted the whole pdf as a picture in word.

Really? I'm pretty sure that's not the way this works.

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by jlink on Saturday February 25, 2017 @08:01AM (#13731378)
thanks for sharing this one, didn't know it.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by andreif on Saturday February 25, 2017 @05:20AM (#13730943)
Yeah, sure, a public one for not privacy-critical PDFs plus something like a Heroku button to build own secure app (with auth and no storage).

See e.g. my file sharing app https://github.com/andreif/SecretFile

Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by akouri on Friday February 24, 2017 @09:05PM (#13729551)
I bet most would, but privacy would be a big concern for me at least. script is optimal format for me
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by marak830 on Friday February 24, 2017 @09:18PM (#13729603)
Privacy is the reason I'd prefer to do this in house.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by a3n on Friday February 24, 2017 @09:50PM (#13729724)
Not to mention corporate privacy/IP.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by marak830 on Friday February 24, 2017 @11:35PM (#13730080)
Indeed. I might also show this to my wife(works in Nissan), might save her a bit of time.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by 2_listerine_pls on Saturday February 25, 2017 @02:28AM (#13730505)
check docparser.com
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by jlink on Saturday February 25, 2017 @08:06AM (#13731399)
interesting service which was not present yet back in 2015 when I wrote my class.
Re:Show HN: PDFLayoutTextStripper – Converts PDF to text whi... by krakaukiosk on Saturday February 25, 2017 @01:13PM (#13732885)
Correct! We launched July 2016