News for hackers. Stuff that matters.
Re:An underrated alternative to Unstructured/Nougat for text... by replicantrose on Tuesday February 06, 2024 @06:14PM (#39282216)
Apache Tika is time-tested and, by some, considered a legacy toolkit. With Tika running as a container and the use of Python bindings, it's possible to get a text extraction experience that is as easy to build with as newer frameworks like Unstructured, but also matches the extraction capability of dedicated extraction models like Nougat. Kind of surprising!

Furthermore, using a backing object store (i.e. MinIO) to hold the source documents is very useful (whether the extracted text is being used for RAG or an LLM training dataset).

There are no comments on this comment yet.