Re:Show HN: Use cookies from Chrome (CDP) in cURL without co...
by ghqst on Saturday April 01, 2023 @09:23AM
(#35400122)
Yeah, you can sometimes find the API or find data sent in JavaScript but not in prerendered HTML, which can save you the pain of headless scraping.
Re:Show HN: Use cookies from Chrome (CDP) in cURL without co...
by juujian on Saturday April 01, 2023 @09:27AM
(#35400155)
I do mean JavaScript. Not sure how many times I have made that mistake... And great advice, that sounds like a neat approach.
Re:Show HN: Use cookies from Chrome (CDP) in cURL without co...
by tomashubelbauer on Saturday April 01, 2023 @12:10PM
(#35401396)
> I have never run into content hidden by Java
Tongue in cheek: You'd never know - servers running Java code generating HTML pages have probably conditionally not-rendered many pieces of HTML that you've never come across in your browsing :)
I have found it's quite easy to snoop on those JavaScript API requests using the Network tab of Chrome Devtools, then copy the network request as a curl command for bash scripts or as JavaScript for browser extensions.