News for hackers. Stuff that matters.

A comment on Web scraping with GPT-4o: powerful but expensive. Parent comment

Re:Web scraping with GPT-4o: powerful but expensive by edublancas on Monday September 02, 2024 @07:42PM (#41429719)
author here: I'm working on a follow-up post. Turns out, removing all HTML tags works great and reduces the cost by a huge margin.
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by 7thpower on Monday September 02, 2024 @10:11PM (#41430581)
What do you mean? What do you use as reference points?
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by edublancas on Monday September 02, 2024 @11:07PM (#41430868)
nothing, I strip out all the HTML tags and pass raw text
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by isaacfung on Monday September 02, 2024 @11:45PM (#41431045)
How do you keep table structure?
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by jaimehrubiks on Tuesday September 03, 2024 @05:14AM (#41432905)
They should probably keep tables and lists and strip most of the rest.
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by AbstractH24 on Tuesday September 03, 2024 @07:00AM (#41433551)
Am I crazy or is there no way to “subscribe” to your site? Interested to follow your learnings in this area.
[ Parent | on HN ]
Re:Web scraping with GPT-4o: powerful but expensive by edublancas on Tuesday September 03, 2024 @02:32PM (#41437703)
there isn't. but you can connect X or LinkedIn.

I might add a subscribe button once I get some time :)

[ Parent | on HN ]