Now Google is using the same content and depriving them of the traffic.
That's pretty scummy in my opinion.
Before:
User: Hey Google, what's the monthly payment on a 20-year mortgage with 3.9% APR?
Google: Oh, MortgateSite would know, go there.
MortgageSite: Here's your calculator.
User: Okay, let me put in the numbers ... awesome, thanks! Oh, neat, they can hook me up with lenders. Let me take a look.
MortgageSite: Thanks for the referral, Google!
Everyone wins.
Today, it's more like:
User: Hey Google, what's the monthly payment on a 20-year mortgage with 3.9% APR?
Google: Oh, that would be this much: ... . Here are some sites where you can dig deeper.
User: Um, okay, might be worth a look. Let's try MortgageSite.
MortgageSite: uhhhhhhh hold on a second. Hey, see our BUY THIS PRODUCT mortgage calculator. AND THIS ONE
User: Uh, okay, I'll just type in--
MortgageSite: HEY! It looks like you're new to this site. Want to get on our mailing list?
User: You know what? Screw it.
MortgageSite: Confound you, Google, for stealing our traffic!
User: Hey Gateway, what's the monthly payment on a 20-year mortgage with 3.9 APR?
Gateway: Here's your calculator, already prefilled with the data taken from your query.
Done.
The problem with all these little sites is, besides bloat, that the "Oh, neat, they can hook me up with lenders. Let me take a look." ends up with user getting malware and/or scammed. The problem with Google is that they can hardly be trusted at this point to do things like this in the interest of users. They want to be the frontend through which you access the Internet.
I used the word "Gateway" in my example as a placeholder; my imaginary perfect world recognizes that things like "currency conversion", "song lyrics" or "mortgage calculator" are data[0], which should be separate from the frontend used to access it. I dream of the Internet where things like these are API-driven and do not involve loading anything other than what's requested - neither ads, nor "value-adds", and definitely not all the rest of the webpage surrounding a mortgage calculator, which is bloat obstructing requested functionality.
--
[0] - yes, even mortgage calculator; it's a mathematical model, an algorithm, and code is data.
You're dreaming of what wolfram alpha is trying to build.
Edit: funnily enough, they parsed the input slightly wrong but you can quickly get to your answer.
Google Assistant (aloud): $3,004
Seems pretty straightforward, no need for a web page.
Actual current results: "Sorry, I don't know how to help with that yet"
While in the interest of "privacy" Google limits the amount of data the banks and others you interact with get from any competing solution, they will sell all of it in Ads Data Hub which is a Google Cloud product. So not so much in the interest of privacy but much more in the interest of Google selling more cloud products.
User: I can afford a livable four walls and a roof from a small amount of savings, without needing to mortgage decades of my future. Brilliant.
The problem with that is that if a poor person can afford a house with cash, then a slightly richer person would just go, "wow, I can buy a really big house! Or two!", and suddenly there is no more space.
Maybe they wouldn't be allowed to buy two, because in "a perfect world", a place to live is a necessity, not a profit vector. Keeping housing supply limited to boost house prices, to appeal to house-owning voters and wealthy landlords, to extract six figures of money from normal people over decades, is nothing like "perfect".
Here I say "landlord epxloitation, profiteering from necessities, houses left unlived in as investment vehicles for the international super rich, NIMBYism and entrenched residents voting against housing stock, AirBNB, broken zoning incentives" and then you say "FREE MARKET it's rich people's right to buy up everything and if you disagree you're dumb, free market is best market". And then we disagree to agree and both leave unhappy. Is that an approximately good summary?
We don't need to determine who gets which for this comment chain, we only need to question whether a 30 year mortgage should be the default way for a normal person to keep the rain off their bed, someone who isn't thinking of a beach condo in Malibu or a NYC penthouse, whether that is the best possible "perfect" world. I think it isn't.
Even then, mortgages aren't something Google can realistically calculate because lenders don't structure loans that simply. There is a fixed rate period, and a variable period after that.
First-time searchers may get suckered in by a personal finance site that relies on ads just as Google itself does, but after that, they'll be using the calculators that are on lenders' own sites when it's time to comparison shop.
Dear every web developer in the world who makes no effort to fight your employers' marketing department on this: I hate you.
let them deal with their own spam, then they'll understand.
Or various Five Eyes addresses.
I think I may make the effort to learn how to make a Firefox extension to automate this. Scrape "about us", "contact", etc. page in background for first email address, then autofill it into the email spam form. Boom, done.
In some places this a growth hacking/marketing tactic (that works), in others it isn't the marketing team but someone up high demanding it because they've seen it. Am still a marketer, but I've been in the latter and trying to fight the good fight.
But, this popup harassment is just going to turn away experienced users. Many of whom will be the educated, well-compensated demographic the site most wants to cultivate. And a subset of that group will have the ability to create or edit blockers, which will then be turned against the offending site. Thus making it more difficult for that company to reach the more desirable users.
That's what I would tell marketing. That wouldn't work for all companies. Some will be thrilled to get a boatload of inexperienced or naive users. But would be nice if the others would start to get worried about what they're losing with their bounce rate.
Take a closer look at the key phrase in my post: "makes no effort". If a web developer who's trying to pay the rent makes even a single comment to the marketing drones about how email signup forms might not be a good idea, and asks if they're sure they want to do it, then that web developer has my sympathy and not my hatred. But if they just cheerfully say "Yes sir!" when the marketing drones make that demand, then I hate them just as much as I hate the marketing drones, because at that point they are marketing drones.
MortgageSite: Please wait while we take 10 seconds to calculate an interest rate AND CHECK OUT THIS SPONSOR!
MortgageSite: Five more seconds. YOUR COMPUTER MIGHT HAVE A VIRUS!! DOWNLOAD THIS FREE SCANNER NOW!
MortgageSite: Your rate has been calculated! Click HERE to reveal it using a totally pointless JavaScript animation!
User: Finally, now I’ll just move my cursor up and close this tab.
MortgageSite: OMFG YOURE GOING AWAY! Please come back and click on me more!!!
I remember a world without lyric sites. These sites provide value. The top links google shows may not because of page layout or bloated sizes but those are the links google presents there are lyric sites with no javascript. Meanwhile tons of mortgage related calculator pages exist on most bank sites and other non-spamming pages.
Clicking a google search result takes seconds to load the link while clicking a ddg link feels much faster. The additional tracking and redirections start to make things feel sluggish.
I remember when Google used to push content developers to be better: no paywalls, no overlays, faster response times, no scammy ads, no content farms.
Those days seem to be over. Google is constantly sending me to content behind paywalls and under pop up content.
Content farms were huge before Google started kicking them out in 2011. Have they made a comeback?
Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?
Have they not? Content marketing seems as alive as ever, if not more.
> Google's latest attempt to keep the crap under control (AMP) is very unpopular on Hacker News, but I guess it shows they are trying to do something?
Because we - myself and many of those other HNers disliking it - believe that with AMP, making web more performant and less crappy is only an excuse, and one that doesn't stand up to scrutiny.
FWIW: I agree that for a long time, the need to rank highly in Google's searches pushed content sites to be a better experience. And the equilibrium we've reached isn't as nice as it was 10 years ago. I just don't think "because Google" really captures the complexity here.
No, I don't blame them for hosting the content themselves for some queries. I never said that.
Just as Amazon holds some responsibility when I purchase a counterfeit item on their platform, it behooves both companies to have higher quality standards -- and both have the market strength to do it.
I found some instructions[1] that says a website can opt out of Google's Featured Snippets with special HTML code:
<meta name=”googlebot” content=”nosnippet”>
[1] https://searchenginewatch.com/2019/03/27/google-featured-sni...
It's not a perfect analogy, because obviously people still hope for referrals from Google. But this idea that because something is published somewhere, it's okay, reminds me of Arthur Dent finding out that the notification about demolishing his house was in the third-subbasement of a local government building in a file cabinet marked "Beware of the Leopard." From rough memory; please forgive if I mis-remembered.
Also going forward you have to add this to your site or I will steal anything that isn't nailed down.
<meta name=”fuzzz4lyfe” content=”notheft”>
I don't know if it's made any difference. Organic search traffic from them has slowly but steadily increased over the years.
Google now sometimes displays a snippet from my competitor's website for searches like "cheapest .io domain" [1]. The snippet seems pretty useless as it doesn't include any registrars' names/links (and my competitor's price info is quite outdated).
In these cases, since the snippet is the 1st thing users see in SERP, and doesn't provide enough info to fully answer the question, I'd wager that my competitor is ultimately receiving the majority of clicks from these snippets.
Appreciate your datapoint. Also btw, when I "view source" the HTML of tld-list.com, I notice it has "nosnippet" inside a comment:
<!--
<meta name="googlebot" content="nosnippet">
-->
IIRC, I eventually removed nosnippet because it caused google to not display microdata in SERP (see the "$25.99 to $99.80" in the above screenshot) that were desirable for my traffic. I instead replaced nosnippet with:
``` <meta name="robots" content="noarchive"> ```
And this seemed to have the same effect as nosnippet, but with the added benefit of my microdata still being displayed in SERP.
Google "chancellor" from the UK and it'll suggest the question "who is the UK chancellor now" which expands to name the previous chancellor (an error Google attributes to gov.uk which would have been updated in July with the new appointment). Click on the link to Google the question and this time the snippet attributed to the same source will be up to date and give you the correct answer. But if I still need to load a new page to get the right answer I'd preferred to go direct to the source page without it serving me wrong answers wrongly attributed first.
And the other questions are also a rabbithole of rubbish to fall down (the question "who was the previous chancellor?" answered by "previous chancellors have opted for whisky..." is my favourite combination) or excerpt the wrong part of the page and then encourage you to search Google again to get the same unhelpful excerpt again rather than deign to look at the source period.
Getting this stuff right across millions of queries and unstructured data of mixed quality is undoubtedly an incredibly hard technical problem but I think it's probably better to sometimes let users leave Google properties than feed them inaccurate answers inaccurately attributed...
I think we need to go back a little and think. When the internet started, if you wanted to monetize your website, and attract users, what would you do? There were no search engines. Nowadays, people claim they're at the mercy of Google, but it's more that their very existence and business model was enabled and made viable by Google. Since it seems, without Google, no one would find them and visit their website. In that sense, I find it hard to say Google is taking things away, it seems more like Google is giving and giving, sometimes, it gives less, and sometimes it gives more.
In the 90s, people were making websites because they wanted to make websites, and some of them were informational, and some of them were just whatever cool thing somebody decided to publish. Keeping track of your favorite sites was harder, but eventually browsers added bookmarks and it became a bit easier.
Okay, now how do you find all of that stuff? Three answers kind of sprung up: sites that are list of sites, directories (Yahoo) and search engines. They kind of work, but finding good stuff is still hard, and while the best answer here is probably search, the indexes aren’t entirely comprehensive and the algorithms aren’t good at sorting good stuff from bad stuff yet, they’re too easily gamed.
Google came along and went with search, but they did two things differently: they built a comprehensive index and they had a good algorithm for sorting good stuff from bad stuff. This didn’t stop an entire industry from springing up to try and game it, but they made gaming their algorithm an increasingly expensive proposition, and just doing the right thing easier.
So now we have a good search engine with a comprehensive index, now the web has a different problem: as much stuff as there is, it turns out there’s not enough stuff about a lot of stuff, and this is obvious once you have a good enough search engine that will till you whether there is enough stuff about your term. Well as it turns out, it is already 2004 and people are just now figuring out how to make lots and lots of money with this internet and web stuff. Google has come out ahead of the pact as the obviously superior choice for searching all this stuff, their competitors get a lot better, but Google stays ahead of them, and they’re able to maintain a lead by simply being so much better there is no reason to stop using Google so long as they’re not evil.
They also figured something else out: people are looking for information, or the tools for getting that information. Who knows more about getting information than peoples whose jobs it is to find and surface the right information? And it turns out that some things you can’t find with just a webpage, you need other kinds of tools, like calculators. So they built them in, because at the end of the day, it’s the information that people actually want, they don’t need the debris that comes along with loading a full webpage.
tl;dr Google is an information services company and always has been. Characterizing them as a search engine or advertising company is a massive understatement and was always inaccurate.
I do queries like "2 tbsp in tsp" or "45 * 22" or "$25 CDN in USD" all the time and I don't think of them as a search.
Same goes for my Echo. I ask it all the time for today's weather or what time the Texas Rangers are playing or who is the starting pitcher for the Giants tonight. None of that feels like a search.
EDIT: added some sources (sorry, that's the best I could dig up in 5 minutes)
[1] https://www.google.com/googlefinance/disclaimer/
[2] https://searchengineland.com/google-now-with-real-time-nhl-h...
[3] https://bits.blogs.nytimes.com/2015/03/31/ibm-scores-a-weath...
1: https://www.wolframalpha.com/input/?i=texas+rangers+next+gam...
"How is Wolfram|Alpha's data checked?
We use a portfolio of automated and manual methods, including statistics, visualization, source cross-checking and expert review. With trillions of pieces of data, it's inevitable that there are still errors out there.
[...]
How is real-time data curated?
Wolfram|Alpha effectively checks real-time data (such as weather, earthquakes, market prices, etc.) against built-in criteria and models. If an unexpected deviation is found, Wolfram|Alpha will normally indicate it, for example by showing lines as dashed."
https://www.theverge.com/2019/6/18/18684211/google-song-lyri...
Genius doesn't own the copyright to the lyrics and screwing with the punctuation doesn't create a new work, so I'm not sure they have much of a case against anybody. Maybe they could come up with a ToS violation or CFAA case against LyricFind?
Genius had to invent a watermarking system for texts: https://betanews.com/2019/06/17/google-genius-com-lyrics/
I do emphathize with the companies being scooped by Google, and so I have concern that they are viable economically and that Google compensates them for getting the info from them.
But in the mean time, it's a success for me when my search gives me the answer immediately instead of two or three rounds of indirection.
Are you saying that there's no excerpt at all above the search results, when you expected one; or that there is an excerpt, but it's not from the most appropriate/relevant source?
It's annoying when there's a page that has all the information you might need, but to find it, you first have to scroll past a bunch of Stack Overflow posts that are more suitable for dilettantes who prioritise trying a quick fix over learning best practices. But when you do make the effort to find the most promising link and your browser allows the search engine to record your choice, you at least get a vote regarding how the ranking should be adjusted. When users just look at the results but don't interact with them any further, that feedback vanishes.
A win for Google isn't necessarily a win for Google users.
If you have been to a local news site recently, you might agree that they are similarly "scummy," overrun with banners, popups and native ads.
no dark patterns, loads quickly, uses fathom for analytics so no tracking. There are affiliate links to LendingTree, but they're just text links and i tried to be non-obtrusive to the ux.
also, have an android app that just hit 5,000 installs since being launched in mid-April: https://play.google.com/store/apps/details?id=io.mortgagecal...
currently ranking #3 for 'mortgage calculator' on google play. pretty proud of that! beating out Zillow and Quicken's mortgage calculators.
For example I was searching for a lease termination letter template. There are sites dedicated to documents, letters, forms and templates. Many of them are pretty spammy or messy. Then I found a page of a real estate portal. They had just what I needed, without superfluous ads with nice UX. It's in their interest to help people to do this, because then they will probably search for a next apartment. They, of course, are fast to offer additional services on top like searching for the move services. But nevertheless, providing this service is something on edge what they are doing - a marketplace for real estate. Only then they will get some money, so the side experience should be as smooth and painless as possible.
Another example is a tax/salary calculator on a job board. Or... most what Google is doing to some extent, but with different trade-offs.
I know it's nothing new, but it was interesting for me to notice it recently so directly.
I though AMP was opt-in for publishers? As in you have to write the front-end spec and pick a CDN to cache with.
It is but Google will only put you in the carousel at the top of the search if you use AMP. That's a significant enough driver of traffic where publishers can't afford to lose it.
we had an emergency AMP project after our traffic dropped 35% due to a smaller competitor implementing AMP and google rewarding them by boosting them in the search results.
So unless you can survive getting bumped off of the first page of results, not implementing AMP isn't optional.
I now use https://github.com/bentasker/RemoveAMP, but that's only possible because I'm lucky enough to have a Jailbroken iPhone. :/
I guess you'd call it an "extension"? It feels weird to use that term, as it implies (to me) that there's some sort of extension framework in place. There isn't, it's just code injection.
Yeah extension does sound weird.
Every link aggregator that indexes your AMP page will cache it, including Bing and Baidu. That's what makes safe prerendering possible.
If the right investments were made, every software company's product could be a feature of Microsoft Windows, and Microsoft Windows could be a feature of Intel CPUs.
A lyrics website recently put fake lyrics on their site to prove Google was copying, and sure enough the fake lyrics showed up a couple weeks later.
Google isn't just taking content from "evil" companies like Yelp, they're doing it to everybody.
Job search, shopping, song lyrics, news, and who knows what else, is all being somewhat blatantly lifted. And nobody can stop it because blocking Google is a death knell to any site
Granted, but the comment you replied to was about the difference between a feature and a company.
I'd rather just type CAD to USD.. or BTC to USD, or any other combination in my address bar than type xe.com, load a bunch of libraries, including Facebook connect. Ghostly reports 6 trackers blocked.
Furthermore, it took 3.45 seconds to load xe.com in Incognito, while google only took 1.1
Or retirement calculator: https://www.daveramsey.com/smartvestor/investment-calculator
New versions of Chromium will begin hiding the full URL entirely, probably so that more and more marketing/targeting related UTM parameters can be jammed in without the user ever even knowing.
Pay to play baby if you want to show up
Monopolization of cultural information is antisocial.
I set up custom search keywords in every browser I use for this, among many other custom searches. In that particular case I query xe.com from an address bar search (not affiliated with them I've just always used the site) as xe <amount> for my most common currency conversion and others like xeyen for Japanese rates, xegbp for British Pound, etc. Takes me right to the page.
Even for various Google services I set them up to save unnecessary clicks.
I generally search for "1000 USD in AUD" as a workaround.
(1000 * incorrectly_rounded) / 1000 == incorrectly_rounded
If I want to look up an actor I'll just add 'IMDB' to the end and get a much better experience on the site itself than google could ever offer in their search page.
Exactly why go to those sites when you can get the same experience without leaving Google? And its not like Google isn't serving the ads on both Google and those "scummy" websites anyway, and no one is going to out dark pattern Google.
I feel the same way about a lot of outbound sites on Google. There are a bunch of things I just don't want to go to another site for. Off the top of my head:
- Exchange rates. Although this one I find infuriating because Google doesn't know how to correctly round off exchange rates (as in, there's a standard). This manifests as, say, showing 2 decimal places for AUD/USD when they should be showing 4.
- Calculator
- Mortgage calculator
- Song lyrics
That sort of thing. If you go to any sites that provide these sorts of things they're typically "scummy". Lots of ads, lots of Javascript, lots of dark patterns to make you load more page views (eg a mortgage calculator that'll mysteriously take 3 steps/page loads to calculate).
I'm glad these are in search results, typically in significantly better versions. And I don't think anyone who runs a site built around a basic formula for interest calculations has any right to complain about it.
Of course this will be painted as "where does it end?" but not every surface is a slippery slope.
Just look at the likes of Yelp who complains about Google "stealing" their content. Well, Yelp is about one of the scummiest businesses out there. So I won't feel sorry for them, not now, not ever.
The one weird case here is AMP. Like I get Google's motivations here. Many companies develop terrible mobile sites that run badly or not at all and AMP IS much faster, generally speaking. Yet still it seems so heavyhanded with seemingly no opt out (on the consumer or publisher side). I don't really understand why Google wants to die on this particular hill.