Minifying HTML for GPT-4o: Just Remove All the HTML Tags

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by cpursley on Friday September 06, 2024 @05:18PM (#41469801)

What I do is convert to markdown, that way you still get some semantic structure. Even built an Elixir library for this: https://github.com/agoodway/html2markdown

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by bearjaws on Friday September 06, 2024 @05:38PM (#41469910)

Seems to be the most common method I've seen, it makes sense given how well LLMs understand markdown.

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by wis on Saturday September 07, 2024 @06:58AM (#41473135)

Why do LLMs understand markdown really well? (besides the simple, terse and readable syntax of markdown)

They say "LLMs are trained on the web", are the web pages converted from HTML into markdown before being fed into training?

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by nprateem on Saturday September 07, 2024 @08:25AM (#41473528)

I think it says in the Anthropic docs they use markdown internally (I assume that means were trained on it to a significant extent).

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by cpursley on Saturday September 07, 2024 @03:00PM (#41475691)

I think Anthropic actually uses xml and OpenAI markdown.

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by ascorbic on Sunday September 08, 2024 @02:54AM (#41478681)

They're trained on lots of code, and pretty much every public repo has markdown in it, even if it's just the README.

[ Parent | on HN ]

Re:Minifying HTML for GPT-4o: Just Remove All the HTML Tags by audessuscest on Saturday September 07, 2024 @06:24AM (#41472953)

I did that with json too, and got better result

[ Parent | on HN ]