News for hackers. Stuff that matters.
Re:A Google bot scrapes pricing info by adding items to carts by whoisjuan on Wednesday July 01, 2020 @02:15PM (#23704639)
This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process.

I always have found that kind of shady but it's probably known to increase conversions.

What I found interesting is that this an open attack vector for e-commerces. Multiple bots can hit a website and start adding items and start the checkout process. This basically creates an unprecedented cart behavior data influx that ruins any possible usage for data coming from legit customers. Maybe cleaning the data wouldn't be that hard but if someone knows what they are doing they can really make it hard (separate IPs, emails and cart behavior)

I doubt Shopify or Magento have anything to prevent this.

Re:A Google bot scrapes pricing info by adding items to carts by mmcconnell1618 on Wednesday July 01, 2020 @03:06PM (#23705177)
Not all shipping charges can be calculated ahead of time. For example, you may offer free shipping on orders over $50. You may charge $9.99 for the first item, $5.99 for each additional item. You may charge by weight of the whole order. You may have oversized items or packages that can be combined to reduce shipping charges. Some items may ship together as OTR Freight, while others can go via the local postal service. Buying multiple items changes this calculation.

So, yes, you can estimate shipping for a single item but you can't always present the per-item shipping charge as it depends on the context of the whole order.

Re:A Google bot scrapes pricing info by adding items to carts by jasonv on Wednesday July 01, 2020 @03:21PM (#23705353)
What this poster said.

Yes, a lot of smaller e-commerce platforms could do this, but finalizing order value can be a very complex workflow for bigger merchants with more varied sku mixes.

I’ve worked in multi-billion dollar Ecom companies where the programs to refine the order checkout process gets scoped as a multi year effort accounting for a couple of decades of legacy cruft... even if you separate the “product/tax/shipping” calculations from the “customer/credit/rewards” dependencies. But it’s often not worth separating them because they’re very inter-dependent. Moreso when you involve drop shipping or made-to-order things.

Re:A Google bot scrapes pricing info by adding items to carts by chrisan on Wednesday July 01, 2020 @03:53PM (#23705756)
How does that change by having the bot add items to the cart? You haven't solved anything

You are still left with the same scenario as if the store listed the individual shipping price on the front page

Google isn't going to know what other items you _might_ add to show you a "real" shipping cost

Re:A Google bot scrapes pricing info by adding items to carts by hrktb on Wednesday July 01, 2020 @04:49PM (#23706487)
I’d assume parent’s point is regarding the “which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process.” part.

There’s a lot of legitimate case were showing shipping price upfront is just not doable or valuable to the customer.

BTW there are a surprising amount of shops for specialized goods that won’t even list the final price at the end. The customer places an order, and they update it with a finalized price after a human looks at the content, and from there the customer is free to pay the transaction or give up the order.

Re:A Google bot scrapes pricing info by adding items to carts by zoomablemind on Wednesday July 01, 2020 @06:13PM (#23707410)
Even the Y2K-style ecommerce stores usually had a separate S&H section for some guidance. These days the H part (handling) seems less in vogue (perhaps still common on ebay), while S part is pretty predictable if not free.

It's the T (taxes) part that may be still a tipping point these days, but it's just between vendor and your state,

Re:A Google bot scrapes pricing info by adding items to carts by hrktb on Thursday July 02, 2020 @03:32AM (#23710829)
We are in agreement that there needs to be explanation on what's going on, and not just "we'll set some price yon won't know why".

In my experience, the most fluctuations were on international shipping by small vendors. Lego bricks for instance, where it makes a big difference if you request 5 small pieces that weight 20g total and can wait 3 months, or if it's 500+g in a middle sized box and you want it in 2 days.

Even with average indication on what to expect, depending on the combination you are requesting the vendor might use a different carrier, different shipping method and so on. They could make it more simple with a range of arbitrary standard fees, but then it costs a lot more to the customer, putting the vendor at a disadvantage price wise. In particular people have visceral reactions to overly high shipping prices.

Re:A Google bot scrapes pricing info by adding items to carts by Xelbair on Wednesday July 01, 2020 @05:49PM (#23707154)
And what's even more interesting - human would do exactly the same thing.

Add items to card, check the total price and then decide whether to buy it(i remember trying to order some stuff form one japanese plamo store - and it didn't provide exact prices before checkout. I went through the process, but even the cheapest option for delivery was way too high - as 2x price of the whole order)

Re:A Google bot scrapes pricing info by adding items to carts by LorenPechtel on Friday July 03, 2020 @12:56PM (#23725550)
I've had that sort of experience with a Japanese store once--the prices were good but the shipping (the only thing they offered was international FedEx) killed it. The US is just as bad--we don't have slow international options.
Re:A Google bot scrapes pricing info by adding items to carts by saltedonion on Wednesday July 01, 2020 @03:59PM (#23705828)
It matter because if you purchase multiple things your average shipping cost per item changes. If you only calculate shipping based on first item shipping cost it will be inaccurate.
Re:A Google bot scrapes pricing info by adding items to carts by mdoms on Wednesday July 01, 2020 @04:05PM (#23705895)
But Google is only showing a result for a single item.
Re:A Google bot scrapes pricing info by adding items to carts by ehnto on Thursday July 02, 2020 @02:34AM (#23710583)
No one said the bot is getting good data. I assume it's trying to get the best possible outcome by adding to the cart, but I doubt it's getting the real final price for many merchants even by doing that.
Re:A Google bot scrapes pricing info by adding items to carts by dheera on Wednesday July 01, 2020 @07:46PM (#23708221)
You could GeoIP the user's IP address and display an initial tax + shipping estimate.
Re:A Google bot scrapes pricing info by adding items to carts by nickip on Wednesday July 01, 2020 @07:52PM (#23708275)
Tax isn't always only region based. You also need to account for Vat Tax which has a bunch of conditions around it too that you can't assume.
Re:A Google bot scrapes pricing info by adding items to carts by dheera on Wednesday July 01, 2020 @10:58PM (#23709478)
Include the statistical average tax in the price, and then up it or down it slightly at check out like a tax return. The user will be happy that the advertised price is a good estimate of the final price. Same for shipping.
Re:A Google bot scrapes pricing info by adding items to carts by clairity on Wednesday July 01, 2020 @03:36PM (#23705538)
that's true that the calculations can get complicated pretty quickly in ecommerce, but google probably has all the data it needs (origin zip, destination zip, likely carrier(s), possibly even the weight/size of each item) to provide a pretty good estimate in most cases. they could even calculate a range for [1 item per box, all items in 1 box].

the important bit is to present it as a separate line item (with grand total) so that consumers can decide how much to trust the estimate.

that would be an even clearer shot across the bow of amazon, walmart, and the like, who provide comparison across their own platform merchants, but not across all merchants everywhere.

Re:A Google bot scrapes pricing info by adding items to carts by gwright on Wednesday July 01, 2020 @06:09PM (#23707382)
Google guessing seems like a terrible idea. That will just confuse the consumer when they go to purchase and find a different value, possibly creating a customer service problem for the vendor through no fault of their own.
Re:A Google bot scrapes pricing info by adding items to carts by abiogenesis on Wednesday July 01, 2020 @09:46PM (#23709103)
True, but you can at least show the total shipping cost for the shopping cart, given a zip code or a similar indication, without completing the whole account creation/checkout process.
Re:A Google bot scrapes pricing info by adding items to carts by three_seagrass on Wednesday July 01, 2020 @09:03PM (#23708825)
Is Google indexing the shipping cost?

I don't see how this is relevant because ecommerce sites will change the price in the cart (or reveal it) before shipping is even calculated.

Re:A Google bot scrapes pricing info by adding items to carts by bhuber on Wednesday July 01, 2020 @08:25PM (#23708528)
Even assuming you already know the customer's shipping address and ignore the multiple-items problem, this is still difficult to accomplish from a computational complexity perspective. Calculating shipping cost is likely at least an order of magnitude more expensive than simply looking up list prices in a database - you have to look up the customer's address, go through a bunch of tax rules, figure out the shipping cost (however that works, I honestly don't know but I assume it's non-trivial), etc. Now consider the fact that prices displayed at checkout make up a tiny fraction of prices that are requested by the site. Every time an item appears anywhere on the site you probably want to display a price with it. So now your infrastructure costs for handling pricing requests go up by an order of magnitude since all of them no require expensive pricing computation, whereas only a tiny fraction did before.

On top of all this, if all you're displaying is list price you can cache that very effectively and significantly reduce the load on your backend, probably by at least another order of magnitude. As with many things, items loaded on ecommerce sites tend to follow a Pareto distribution, for which caching is very effective. Adding a shipping address to the mix will destroy this caching ability, so not only are your requests 10x more expensive, 10x more of them now make it to the backend. There are various tricks you can do to try to have your cake and eat it too, but none of them are easy or simple. At the end of the day, while this is definitely a useful and desirable feature for customers, it has significant cost for both development time and hardware.

TL;DR this is actually a much more difficult technical problem to solve cost effectively at scale than it initially appears.

Re:A Google bot scrapes pricing info by adding items to carts by mmcconnell1618 on Wednesday July 01, 2020 @09:01PM (#23708811)
In many cases, this is not only computationally expensive on the server, it also requires one or more requests to external APIs which further slows things down. Imagine needing to query the API of your tax vendor, then also your shipping provider of choice for every single item displayed on a page. Even if you did this client side, asynchronously, it would be a lot of extra requests for something that most shoppers won't even pay attention to.

I ran an ecommerce platform company for many years and you had merchants with very complex shipping and tax schemes, or you had merchants that made it super simple with a basic rate table. The complex merchants had margin on every order at the cost of processing external API calls. The simple rate table merchants had great margin on some, lost money on others but were happy with their average shipping margin.

Re:A Google bot scrapes pricing info by adding items to carts by gnuisance on Wednesday July 01, 2020 @10:02PM (#23709194)
Having worked with a major eCom platform as well, this is exactly the standard case. Both shipping and tax are complex problems which do not have a simple solution for scraping by a search engine.

Shipping is often highly dependent on the location of the buyer and often involves full estimate calls from each carriers APIs (USPS, FedEx, UPS). The only major data point I would focus on is whether the shipping is free or flat rate.

Tax is even more complicated. Merchants often outsource tax calculations to a third-party service such as Avalara, which calculates unbelievably complex taxing schemes even down to the zip code, as tax laws are becoming increasingly more complex.

Because of these reasons taxes and shipping are not widely useful data points for search engines. That may change in the future, however. I could imagine it becoming another SEO topic to be accounted for, similar to meta tags on product pages.

Re:A Google bot scrapes pricing info by adding items to carts by bluntfang on Wednesday July 01, 2020 @03:45PM (#23705659)
Well we can change this by including shipping as the total price and not give deals on shipping. Deals on shipping are dark patterns.
Re:A Google bot scrapes pricing info by adding items to carts by innocenat on Thursday July 02, 2020 @01:01AM (#23710155)
I don't understand. So if the item is $5 and shipping is $5 regardless of the number of items being bought, then if I bought 2 I should pay $20 instead of $15?
Re:A Google bot scrapes pricing info by adding items to carts by bluntfang on Thursday July 02, 2020 @12:07PM (#23714776)
What don't you understand about deals on shipping being dark patterns?
Re:A Google bot scrapes pricing info by adding items to carts by innocenat on Thursday July 02, 2020 @12:52PM (#23715221)
It seems that you are saying that properly charging for shipping to be a dark patterns? Or you mean different kind of deal? Can you elaborate? As far as I know, the shipping charge doesn't scale linearly with number of items, so to me including shipping within the items price is going to overcharge the customer.
Re:A Google bot scrapes pricing info by adding items to carts by gruez on Wednesday July 01, 2020 @02:25PM (#23704734)
> This bot is simply trying to get the final price (with tax and shipping) which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process.

It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

Re:A Google bot scrapes pricing info by adding items to carts by stronglikedan on Wednesday July 01, 2020 @02:42PM (#23704929)
I just used a website that had a simple form at the bottom of the cart. It had one text input for the postal code, and a button to get the rates to that postal code based on what was in my cart. IMHO, this is how it's done right, since all you need to know is general location and weight.
Re:A Google bot scrapes pricing info by adding items to carts by jiofih on Wednesday July 01, 2020 @02:56PM (#23705085)
Maybe I haven’t done enough online shopping recently, but as far back as I remember this used to be the norm: enter postcode to calculate shipping, get precise final price without even adding to cart. Is it not the case anymore?
Re:A Google bot scrapes pricing info by adding items to carts by the_pwner224 on Wednesday July 01, 2020 @03:39PM (#23705566)
On many websites it still is. But recently many smaller independent stores use the Shopify platform where shipping is the penultimate step, before billing. You have to give address, email(!), phone number(!, mandatory), etc. before getting the price. I normally just use a fake email and number to get shipping price, and then do the actual checkout in incognito. Pretty sure if you do enter your real info and don't continue with the purchase then you'll get email spam telling you to buy stuff.
Re:A Google bot scrapes pricing info by adding items to carts by fwip on Wednesday July 01, 2020 @07:57PM (#23708310)
Also increasing conversion rates via the sunk-cost fallacy. If I see up front that the air conditioner I'm ordering costs $30 to ship, I'll check another site. But if I already decided on this one and I just did all the work to go find my credit card, enter my billing info - maybe I'll just say "ehhh. Fine." and purchase it anyways.
Re:A Google bot scrapes pricing info by adding items to carts by jimktrains2 on Wednesday July 01, 2020 @06:01PM (#23707292)
Phone number is required or otherwise highly encouraged by some shippers, like FedEx.
Re:A Google bot scrapes pricing info by adding items to carts by gruez on Wednesday July 01, 2020 @04:19PM (#23706069)
I'm pretty sure most sites (eg. Amazon) does the same thing. Probably for the reason you mentioned: so they can have your contact information to send you spam later.
Re:A Google bot scrapes pricing info by adding items to carts by datenhorst on Thursday July 02, 2020 @04:50AM (#23711259)
My girlfriend uses this as a pretty effective tactic to get discounts - you just have to wait a couple of days and they'll send you an email with a lower price to keep down abandonment rates
Re:A Google bot scrapes pricing info by adding items to carts by judge2020 on Wednesday July 01, 2020 @09:48PM (#23709117)
Haven't received any "items in your cart" emails from Amazon peronally, but I will end up seeing Ads for the products later on.
Re:A Google bot scrapes pricing info by adding items to carts by mkl on Wednesday July 01, 2020 @05:14PM (#23706783)
Is that US only? You need country too.
Re:A Google bot scrapes pricing info by adding items to carts by rietta on Wednesday July 01, 2020 @05:28PM (#23706937)
For commercial scale products you need more than that. The full address. Is the address residential vs commercial with a loading dock? That and more factors impact the shipping price a lot! Logistics companies have people who have to research an address and look at Google Earth photos of the property to answer these questions.
Re:A Google bot scrapes pricing info by adding items to carts by Ruthalas on Wednesday July 01, 2020 @04:59PM (#23706610)
I will bail from the purchasing process on sites that are unwilling to give me a final price before I enter payment information.

A postal code should suffice, and I'm not providing more personal information if the site is unwilling to say what I'll be charged upfront.

Re:A Google bot scrapes pricing info by adding items to carts by texasbigdata on Wednesday July 01, 2020 @06:21PM (#23707468)
Isn't that most of them? Unless you assume they eat variable shipping by 'fronting' you a fixed price. Apologies if I'm misunderstanding what you're saying.
Re:A Google bot scrapes pricing info by adding items to carts by Ruthalas on Wednesday July 01, 2020 @06:52PM (#23707781)
It is still typical (for which I am thankful), but that seems to be shifting a bit.

A relatively common example would be shops using the Shopify platform.

Re:A Google bot scrapes pricing info by adding items to carts by whoisjuan on Wednesday July 01, 2020 @02:28PM (#23704764)
Yeah that's true. But from a UX perspective there are ways to make this less opaque. Perhaps a call to action at the top of the listing with an entry box to enter the Zip Code so an approximate final price can be calculated.

Any good UX designer can come up with a solution for this in a couple of hours or less. There's just no motivation to make it happen because this obfuscation of data is particularly optimized and useful for the sellers.

Re:A Google bot scrapes pricing info by adding items to carts by jeanvalmarc on Wednesday July 01, 2020 @03:22PM (#23705375)
Amazon just lists it as a "subtotal," which I think is probably the best way to do it. I don't know about the "UX designer in a couple of hours" line: automatic shipping is a nightmarish bag of worms and isn't really a UX problem. What do you do if they order multiple SKU's that don't pack nicely into one box or are warehoused in different locations? Or if there is something weird about their shipping address? What do you use for box size and weight, and what approximation did you use for dunnage?

You can approximate UPS/FedEX costs by fitting a trend line, they are decently modeled by a linear (base charge + K*distance) function, but when you go to buy the label you might be way off. This puts you in the lose-lose-lose of either eating the difference of wrong estimates, overcharging for shipping and losing conversions, or just making people hate you by increasing the shipping over the estimate. Making a shipping API call is noticeably slow, so most people require an interaction after the address is entered.

Tim Sweeney's hot take was that "the two hardest problems in computer science are cache invalidation and shopping carts!"

Re:A Google bot scrapes pricing info by adding items to carts by neutronicus on Wednesday July 01, 2020 @02:36PM (#23704852)
e-commerce sites seem to be asking for my location all the time anyways.

If they're doing that anyways, they should have everything they need to hazard a pretty good guess (and then they have an actual inducement for me to provide it).

Re:A Google bot scrapes pricing info by adding items to carts by whoisjuan on Wednesday July 01, 2020 @02:40PM (#23704907)
Agree!
Re:A Google bot scrapes pricing info by adding items to carts by TeMPOraL on Wednesday July 01, 2020 @02:45PM (#23704956)
ZIP code box that doesn't oblige you to provide any more accurate data, and also without it, it should still be possible to display brackets. "Shipping: $6 - $24, [enter ZIP code for detailed quote]".
Re:A Google bot scrapes pricing info by adding items to carts by wdb on Wednesday July 01, 2020 @03:39PM (#23705565)
They could at least include the tax in the price. That's normally fixed depending on the item category e.g. low or high tax rate. Only the US is being weird with its taxes.
Re:A Google bot scrapes pricing info by adding items to carts by ww520 on Wednesday July 01, 2020 @03:46PM (#23705663)
Tax is depending on where the buyer is coming from.
Re:A Google bot scrapes pricing info by adding items to carts by kube-system on Thursday July 02, 2020 @01:41AM (#23710351)
Canada has provincial sales taxes.

It's really not that weird when you consider the US is not a unitary state.

Re:A Google bot scrapes pricing info by adding items to carts by ehsankia on Thursday July 02, 2020 @01:18AM (#23710247)
I don't know if this is true, but I've been many websites that actually claim that they have an extra deal they can't show you until you put the item in the cart. I used to see those a lot a few years ago, not sure if it was a real legal thing or just a trick to get people to add it, but that was definitely a thing and not related to tax+shipping.
Re:A Google bot scrapes pricing info by adding items to carts by kube-system on Thursday July 02, 2020 @01:24AM (#23710277)
It is a contractual obligation with the product manufacturer.

Something like: we will allow you to sell our product but only if you don't advertise discounts more than x%.

Re:A Google bot scrapes pricing info by adding items to carts by enriquto on Wednesday July 01, 2020 @04:01PM (#23705848)
> It's usually not possible because you don't know how much the shipping + taxes are until the customer enters the billing information.

Sure, they show you a different content depending on your IP address and lots of shady heuristics, but when it comes to estimate a shipping cost, it is absolutely impossible: you can just be anywhere, who knows where you are. I say bullshit.

Re:A Google bot scrapes pricing info by adding items to carts by miohtama on Wednesday July 01, 2020 @05:44PM (#23707100)
All EU prices must be tax included.

All hospitality prices must include cleaning and service fees.

It is only the US were hidden fees and charges may apply and the price regulation, if any, is more tilted towards corporations.

Only upfront shipping charge is tricky because it depends on so many factors.

Re:A Google bot scrapes pricing info by adding items to carts by shultays on Wednesday July 01, 2020 @06:21PM (#23707466)
I find not including tax better tbh. That way people are being reminded how much tax they are with every purchase.
Re:A Google bot scrapes pricing info by adding items to carts by input_sh on Wednesday July 01, 2020 @07:21PM (#23708017)
Every EU bill should come with the price without VAT (value-added tax) and with the VAT applied, leading to the same result.

The only difference is that the VAT is a certain % depending on your country (mostly 15-25%), making it easy for a merchant to calculate it with a single data point. Percentage is also always a fixed round number, so you can calculate it in your head when you stumble upon "+ VAT" somewhere without providing that data point to the merchant for calculation purposes.

Re:A Google bot scrapes pricing info by adding items to carts by shultays on Thursday July 02, 2020 @01:14AM (#23710221)
Yea, you still can get the info but how many people do really care. I am just saying the people would be more aware on how much they are being taxed. Imo making it obvious like US does would be more helpful even if it is not as convenient

I used to live in Turkey, where taxes on some stuff are insane but most people won't ever know because it is not as obvious. In a similar vein, income tax is also quite hidden in Turkey, at least from yhe perspective of employee

Re:A Google bot scrapes pricing info by adding items to carts by bcrosby95 on Wednesday July 01, 2020 @10:10PM (#23709224)
Yeah. It's easy in the EU.

My favorite example of how absurd it can get in the US: one side of my friend's street has 9.5% sales tax, the other side has 7.25%. same state, city, and zip.

Re:A Google bot scrapes pricing info by adding items to carts by chokma on Thursday July 02, 2020 @03:09AM (#23710733)
Your are right, normally VAT is a fixed integer value. But that is not guaranteed, historically we had decimal values, too. (My father encountered this once at IBM in the time of punch cards - turns out, having to change this on machines with limited RAM is quite difficult, akin to the Y2K problem.)
Re:A Google bot scrapes pricing info by adding items to carts by adwww on Thursday July 02, 2020 @05:58AM (#23711613)
The UK was 17.5% VAT for years.
Re:A Google bot scrapes pricing info by adding items to carts by lmm on Wednesday July 01, 2020 @11:14PM (#23709585)
We don't break out how much the purchase price was reduced by the use of public goods, so it seems like breaking out the tax would be more misleading than informative.
Re:A Google bot scrapes pricing info by adding items to carts by crazygringo on Wednesday July 01, 2020 @05:17PM (#23706821)
Not only that, but certain brands on certain sites won't show the price, with that message "add to cart to see the price!"

I've heard varying explanations as to why, but at the end of the day it doesn't matter. Adding to the cart is the only way to scrape the price.

Re:A Google bot scrapes pricing info by adding items to carts by cglace on Wednesday July 01, 2020 @03:16PM (#23705287)
How do you show a final price if you don't have all the information needed: tax locality, shipping preference, total cart value discounts etc.
Re:A Google bot scrapes pricing info by adding items to carts by alpha_squared on Wednesday July 01, 2020 @03:21PM (#23705342)
If I'm remembering right, Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were (or if Best Buy was just taking losses on certain items).
Re:A Google bot scrapes pricing info by adding items to carts by thaumasiotes on Wednesday July 01, 2020 @03:26PM (#23705410)
> Best Buy used to have "deals" on items that they "couldn't show you" until item was in cart. They may still be doing this. Best Buy's justification for it was that its agreements with manufacturers prevented it from displaying items below certain prices on their site. I'd never seen this elsewhere to know how pervasive these agreements were

This was also pretty common on Amazon.

Re:A Google bot scrapes pricing info by adding items to carts by hundchenkatze on Wednesday July 01, 2020 @03:42PM (#23705603)
Newegg still does this. Here's an item from their 4th of July sale.

https://www.newegg.com/p/2AM-008Y-00003?Item=9SIAEG2BMZ4393

Re:A Google bot scrapes pricing info by adding items to carts by Spivak on Wednesday July 01, 2020 @04:45PM (#23706414)
KitchenAid is famous for these kinds of agreements as a measure to ensure the public perception of their value doesn’t go down whenever there are sales.
Re:A Google bot scrapes pricing info by adding items to carts by randomdude402 on Wednesday July 01, 2020 @09:17PM (#23708911)
I used to build and manage ecomm sites. We had several manufacturers/brands that we had to agree not to openly display a retail price below a certain amount.

Incidentally, for almost every brand in our industry, that number was 1.8X listed wholesale.

We could sell for less, but not list a price of less. Implementing "add to cart to see price" was good enough at the time to keep them happy.

Re:A Google bot scrapes pricing info by adding items to carts by whoisjuan on Wednesday July 01, 2020 @03:51PM (#23705721)
They still do it.
Re:A Google bot scrapes pricing info by adding items to carts by bronzeage on Wednesday July 01, 2020 @02:27PM (#23704745)
Makes you wonder whether the smart thing to do is just make it convenient for the bots to get the info out so they won't ruin your data and waste your bandwidth. Can't fight them, join them. Can't really stop free information flow.
Re:A Google bot scrapes pricing info by adding items to carts by WWLink on Wednesday July 01, 2020 @02:47PM (#23704978)
Or they could stop spying on their customers and trying to figure out how to add dark patterns to maximize "engagement" and "conversion" lol.

I mean yeah, I get that if you inadvertently make the checkout button hard to find, you'll lose potential sales, but I don't think you need intricate data about what your customers are doing to figure that out.

Re:A Google bot scrapes pricing info by adding items to carts by bduerst on Wednesday July 01, 2020 @10:57PM (#23709473)
Not sure about dark patterns, but you're talking about magnitudes of tens if not hundreds of thousands of visitors here.

Increasing conversion rate by even a few percentage points has huge revenue implications.

Re:A Google bot scrapes pricing info by adding items to carts by stingraycharles on Wednesday July 01, 2020 @03:51PM (#23705723)
They already do that, there’s all kinds of XML and JSON and whatnot standards to communicate product info, inventory, whatnot. The reason Google is doing this is because this information cannot be trusted all the time, there will always be bad actors.

The process may eventually evolve in a cat-and-mouse game, where malicious e-commerce sites try to detect these Google crawlers and serve different price info to them, but let’s hope it doesn’t get this far.

Re:A Google bot scrapes pricing info by adding items to carts by viraptor on Wednesday July 01, 2020 @04:09PM (#23705929)
Given the possibility you get detected and it impacts your organic search ranking... I'm not sure any serious vendor would risk it. And if they do, let them burn.
Re:A Google bot scrapes pricing info by adding items to carts by kqr on Wednesday July 01, 2020 @02:29PM (#23704769)
Or quietly detect the bots and feed them junk data after they've gone through the hoops. Not saying it's the better option, but knowing the business maybe the more likely one.
Re:A Google bot scrapes pricing info by adding items to carts by inetknght on Wednesday July 01, 2020 @02:37PM (#23704865)
> Or quietly detect the bots and feed them junk data

Just a few customers with alternative browsers being detected as "bots" will poison your reputation and income stream.

Re:A Google bot scrapes pricing info by adding items to carts by rabanne on Wednesday July 01, 2020 @05:18PM (#23706838)
No, it definitely won’t. In fact our card payment system has been updated to think alternative os and browsers are high-risk and even decline payment. We didn’t have any revenue loss.
Re:A Google bot scrapes pricing info by adding items to carts by bduerst on Wednesday July 01, 2020 @10:59PM (#23709483)
That's why bots now mimic users beyond user agents, even going so far as loading page assets and javascript. Unless you're using something like recaptcha V3, it's going to be difficult to detect them, and even that requires some interactions first.
Re:A Google bot scrapes pricing info by adding items to carts by Spivak on Wednesday July 01, 2020 @04:43PM (#23706395)
As someone who uses an alternative browser… I kinda doubt it. As a group we wouldn’t even move the analytics needle let alone revenue.
Re:A Google bot scrapes pricing info by adding items to carts by randomdude402 on Wednesday July 01, 2020 @11:09PM (#23709557)
Nah. Google doesn't even let people log into their own email with alternative browsers, and they are doing fine.
Re:A Google bot scrapes pricing info by adding items to carts by toomuchtodo on Wednesday July 01, 2020 @02:32PM (#23704809)
Which is why it should be a chrome extension (like Honey); exfiltrate the data out while providing the end user financial benefits. Messing with the data breaks the user experience and impacts revenue of the target site.
Re:A Google bot scrapes pricing info by adding items to carts by ehnto on Thursday July 02, 2020 @02:30AM (#23710567)
There's a lot that goes into when and where you can show the final item price.

Assuming a simple product, you don't know where the user lives so you can't apply the correct taxes yet. In AUS this is easy because it's the same nationally. But in the US there are dozens of tax combinations that could be applied depending on the location.

Shipping obviously depends on where you live and what you're buying, and few places charge per item shipping these days so it doesn't even make sense to include shipping in a single item cost.

Then you have customer group discounts, some customers get different pricing when they are logged in, even item combinations in the cart can have different prices, you get the idea. It is usually not possible to calculate ahead of time.

Re:A Google bot scrapes pricing info by adding items to carts by alasdair_ on Wednesday July 01, 2020 @06:09PM (#23707381)
>which is ridiculous because e-commerce storefronts should do that in the first place without going through the whole checkout process.

On top of the other reasons mentioned, the seller may have a contract with the manufacturer that covers a "minimum advertised price"

"The FTC says that the price displayed in a secure or encrypted shopping cart isn’t subject to MAP because it’s technically not advertising."

(https://www.thebalancesmb.com/what-is-minimum-advertised-pri...)

Re:A Google bot scrapes pricing info by adding items to carts by mkl on Wednesday July 01, 2020 @05:20PM (#23706857)
It seems like it will also mess up item availability information. If an item has limited stock, bots adding it to carts could make it appear out of stock to real customers.
Re:A Google bot scrapes pricing info by adding items to carts by randomdude402 on Wednesday July 01, 2020 @11:07PM (#23709544)
Most sites don't subtract it from inventory until it's on a completed order.

The main exceptions I can think of are venue and plane tickets, and hotel rooms. These might put a hold on a specific piece of inventory for a short time. They usually tell you when they do.

Re:A Google bot scrapes pricing info by adding items to carts by thawkins on Thursday July 02, 2020 @09:56PM (#23720345)
As a developer of automotive ecommerce sites, it is very common for sites to list a higher price 0h the catalog, and show the true (generaly lower) price in the cart. This is becuase the businessmodels are highly margin sensative, and competative pricing can have big impact, so its a measure to try and mask real pricing.
Re:A Google bot scrapes pricing info by adding items to carts by lalos on Wednesday July 01, 2020 @03:45PM (#23705654)
Also, seems like a clever hack for automated scraping after all most carts are pretty uniform in their structure.
Re:A Google bot scrapes pricing info by adding items to carts by sp332 on Wednesday July 01, 2020 @02:24PM (#23704723)
e-commerce storefronts should do that in the first place without going through the whole checkout process

Yes but how would you verify this or hold them accountable?

Re:A Google bot scrapes pricing info by adding items to carts by gowld on Wednesday July 01, 2020 @02:21PM (#23704699)
There's a tiktok meme doing this to harass the Trump campaign's online store.