Show HN: POC to scrape and structure HTML into JSON for RAG

Re:Show HN: POC to scrape and structure HTML into JSON for RAG by mahi_novice on Tuesday April 29, 2025 @05:22PM (#43838177)

Do you mind sharing more about the implementation details? Any safeguards you have for the urls and all?

[ Parent | on HN ]

Re:Show HN: POC to scrape and structure HTML into JSON for RAG by nirvanist on Tuesday April 29, 2025 @07:25PM (#43839269)

Basically, I use a headless Chromium with Puppeteer to render the page. Then, some logic extracts and cleans the HTML content. Finally, I use Gemini with a specific schema to return a JSON response.

[ Parent | on HN ]

Re:Show HN: POC to scrape and structure HTML into JSON for RAG by mahi_novice on Wednesday April 30, 2025 @01:50PM (#43848591)

Okay, thanks!

[ Parent | on HN ]