News for hackers. Stuff that matters.
Re:How to Run SQL on PDF Files by msravi on Friday February 22, 2019 @12:57AM (#19223459)
So the tool is being used to extract text and then a regex is used to extract the relevant fields/values. Seems that pdftotext[1][2] with awk can do the job on your local machine without uploading your docs.

1. brew install pkg-config poppler (on mac)

2. sudo apt-get install poppler-utils (on Debian/Ubuntu)

Re:How to Run SQL on PDF Files by dexcs on Friday February 22, 2019 @02:07AM (#19223676)
Analyzing the text is the problem. Not extracting. Are there any good open source libs out there?
Re:How to Run SQL on PDF Files by msravi on Friday February 22, 2019 @02:24AM (#19223729)
Sure. But the tool posted here doesn't do that. It merely extracts text, and the "analysis" is a couple of regexes that are tailor-made for that particular pdf. Awk can do that much and a lot more.

If you want to extract tables from a pdf, there's Tabula[1], but it isn't automated to run over the whole pdf - you've to do a manual rectangular selection around the table you want to extract.

1. https://github.com/tabulapdf/tabula

Re:How to Run SQL on PDF Files by mirimir on Friday February 22, 2019 @05:27AM (#19224316)
Indeed. Many years ago, I "ran SQL" on a couple decades of Usenet newsgroup data. Extraction and manipulation involved a bunch of grep, sed, tr and awk (and millions of tmp files). But, as with PDFs of utility bills, it was very specific regex.
Re:How to Run SQL on PDF Files by kwadhwa on Friday February 22, 2019 @01:52PM (#19227976)
Hey, Kshitij from Rockset here.

With Rockset you can avoid ETL when it comes to extracting and manipulating the data. Also, the main value here is that you can join this data with other data sets that are in JSON, CSV, XLS or Parquet formats using SQL to help in analysis.

Re:How to Run SQL on PDF Files by mirimir on Saturday February 23, 2019 @09:57AM (#19233545)
Maybe you could add modules for extracting and manipulating data from popular sources. Such as the most popular social media. Also Amazon, Craigslist, Ebay, etc. And the main search engines.

There are many people who want usable data from such sources. And your service wouldn't be doing any scraping, so you'd probably be OK legally. But IANAL, so do check.

Re:How to Run SQL on PDF Files by danzig13 on Friday February 22, 2019 @07:59AM (#19224895)
I’ve been impressed with Camelot for PDF tables