Show HN: Crul || Query Any Webpage or API

Re:Show HN: Crul || Query Any Webpage or API by nine_k on Tuesday February 28, 2023 @06:43PM (#34976576)

Do you have an easy way to transform e.g. a typical Amazon product page into nicely structured data? Not that it's trying to be very video-gamey.

[ Parent | on HN ]

Re:Show HN: Crul || Query Any Webpage or API by 1vuio0pswjnm7 on Tuesday February 28, 2023 @08:18PM (#34977485)

What should the structure look like. If it is CSV what are the columns.

Taking a quick look at the Amazon site these product pages appear to be enormous in size. Interestingly, the website requires a "viewport-width" header. Otherwise one gets directed to a CAPTCHA.

The page already has some structered data in the form of JSON, including keys like

"title":"xxxxxxxxxx" "displayPrice":"$000.00" "priceAmount":000.00 "currencySymbol":"$" "integerValue":"000" "decimalSeparator":"." "fractionalValue":"00" "symbolPosition":"left" "asin": "xxxxxxxxx" "asin":"xxxxxxxxxx" "acAsin":"xxxxxxxxxx" "buyingOptionTypes":["NEW"] "productAsin":"xxxxxxxxxx" "mediaAsin":"xxxxxxxxxxx" "parentAsin":"xxxxxxxxx" "asinList":"xxxxxxxxxx"

[ Parent | on HN ]

Re:Show HN: Crul || Query Any Webpage or API by nine_k on Tuesday February 28, 2023 @08:42PM (#34977684)

The output may be a collection of CSV files, or a JSON file with nicely structured data, because the page certainly has a pretty visible structure, with various data blocks.

[ Parent | on HN ]