News for hackers. Stuff that matters.

srijansriv

Ask HN: How do we benchmark data extraction, analysis and subsequent enrichment

I have been working on this system that essentially has to extract data from a bunch or structured and unstructured sources, analyse relevance and correctness of the data (im sorry this part is vague) and using the data obtained to fill forms and create write-ups. i do not want to mindlessly use any llm. i want to improve with proof for each of those (broadly, 3) steps. how do we do that? if...

( Read More... | no comments | on HN )
Re:Multimodal Neural Databases on Tuesday March 19, 2024 @08:49AM (#39755169)
This, specially the nlp part, strikes me as a primitive version of the RAG-LLM pair
Re:Algorithms for Modern Hardware on Saturday March 16, 2024 @02:20AM (#39723774)
obligatory mention of cp-alogs: https://cp-algorithms.com/index.html
Re:NLP Algorithms for Clustering AI Content Search Keywords on Wednesday February 21, 2024 @02:31AM (#39451063)
the first thing that comes to mind is CLIP: https://github.com/openai/CLIP maybe combining it with some sort of knowledge graph is what you're looking for