Re:Show HN: Crul || Query Any Webpage or API
by 1vuio0pswjnm7 on Tuesday February 28, 2023 @08:18PM
(#34977485)
What should the structure look like. If it is CSV what are the columns.
Taking a quick look at the Amazon site these product pages appear to be enormous in size. Interestingly, the website requires a "viewport-width" header. Otherwise one gets directed to a CAPTCHA.
The page already has some structered data in the form of JSON, including keys like
Re:Show HN: Crul || Query Any Webpage or API
by nine_k on Tuesday February 28, 2023 @08:42PM
(#34977684)
The output may be a collection of CSV files, or a JSON file with nicely structured data, because the page certainly has a pretty visible structure, with various data blocks.
Taking a quick look at the Amazon site these product pages appear to be enormous in size. Interestingly, the website requires a "viewport-width" header. Otherwise one gets directed to a CAPTCHA.
The page already has some structered data in the form of JSON, including keys like
"title":"xxxxxxxxxx" "displayPrice":"$000.00" "priceAmount":000.00 "currencySymbol":"$" "integerValue":"000" "decimalSeparator":"." "fractionalValue":"00" "symbolPosition":"left" "asin": "xxxxxxxxx" "asin":"xxxxxxxxxx" "acAsin":"xxxxxxxxxx" "buyingOptionTypes":["NEW"] "productAsin":"xxxxxxxxxx" "mediaAsin":"xxxxxxxxxxx" "parentAsin":"xxxxxxxxx" "asinList":"xxxxxxxxxx"