Show HN: Beyond text splitting – improved file parsing for LLM's

Re:Show HN: Beyond text splitting – improved file parsing fo... by Oras on Monday April 08, 2024 @03:37AM (#39967182)

How accurate is table detection/parsing in PDFs? I found this part the most challenging, and none of the open-source PDF parsers worked well.

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by saliagato on Monday April 08, 2024 @04:03AM (#39967320)

worked 100% of the time for me

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by filkin on Monday April 08, 2024 @07:09AM (#39968481)

which software?

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by passion__desire on Monday April 08, 2024 @06:23AM (#39968209)

Have you checked Surya ?

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by Oras on Monday April 08, 2024 @10:28AM (#39970093)

I did and I had issues when tables had mixed text and numbers.

Example:

£243,234 would be £234,

Or £243 234

Or £243,234 (correct).

Some cells weren't even detected.

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by passion__desire on Saturday April 13, 2024 @12:14PM (#40024105)

https://github.com/deepdoctection/deepdoctection

Have you tried this ?

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by verdverm on Monday April 08, 2024 @08:17AM (#39968917)

I've been using camelot, which builds on the lower python pdf libraries, to extract tables from pdfs. Haven't tried anything exotic, but it seems to work. The tables I parse tend to be full page or the most dominant element

https://camelot-py.readthedocs.io/en/master/

I like Camelot because it gives me back pandas dataframes. I don't want markdown, I can make that from a dataframe if needed

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by serjester on Monday April 08, 2024 @09:02AM (#39969281)

Author here. Optionally we implement unitable which represents the current state of the art in table detection. Camelot / Tabelot use much simpler, traditional extraction techniques.

Unitable itself has shockingly good accuracy, although we’re still working on better table detection which sometimes negatively affects results.

[ Parent | on HN ]

Re:Show HN: Beyond text splitting – improved file parsing fo... by xyzjgf on Monday April 08, 2024 @08:32PM (#39975157)

is this the unitable you mentioned https://github.com/poloclub/unitable

[ Parent | on HN ]