News for hackers. Stuff that matters.
Re:How did Google get so big? by wallflower on Friday July 27, 2018 @02:25PM (#17627998)
If I could give a six word response for the technical side: "Jeff Dean, Sanjay Ghemawat and team".

To paraphrase some Cal/UC Berkeley professor that I cannot remember, he said that the problem with scaling is that when you go up a magnitude or more, you may need a qualitatively different solution for the same problem. Jeff Dean and Sanjay Ghemawat and their team have time and time again managed to design and build state-of-the-art, qualitatively different solutions (MapReduce, BigTable, Spanner). And now their team's attention is on AI...

2016 (might be more recent ones) update on AI at Google http://highscalability.com/blog/2016/3/16/jeff-dean-on-large...

Video of full lecture: https://www.youtube.com/watch?v=QSaZGT4-6EY

EDIT: Sanjay Ghemawat and Jeff are like the dynamic duo of large scale systems engineering.

Re:How did Google get so big? by hinkley on Friday July 27, 2018 @05:00PM (#17629364)
> the problem with scaling is that when you go up a magnitude or more, you may need a qualitatively different solution for the same problem.

Never been to Berkeley but I've heard something similar enough times that I've adopted it as one of my own aphorisms.

When the order of magnitude changes, so do the good solutions.

(This is also my thesis for why we keep repeating software history. We stop using a solution because it doesn't work with the current ratio of speed or costs for network, memory, storage and processing. Then someone introduces 10GB networking or SSD disks and the old solution works for 5 more years until someone introduces a new CPU that scales better horizontally or vertically and we start over.)

Re:How did Google get so big? by zjaffee on Friday July 27, 2018 @02:56PM (#17628267)
I think it's important to mention Sanjay Ghemawat is very likely just as important to google as Jeff Dean, just Jeff has been made the public figure for a variety of reasons.
Re:How did Google get so big? by pk2200 on Friday July 27, 2018 @05:04PM (#17629395)
Urs Hölzle (engineering VP) and Eric Veach (AdWords) also don't get nearly as much recognition as they deserve.
Re:How did Google get so big? by int3 on Friday July 27, 2018 @07:39PM (#17630398)
Huh, I had no idea the guy who wrote the light transport thesis also built Adwords.
Re:How did Google get so big? by mrdmnd on Friday July 27, 2018 @07:44PM (#17630437)
He's also the chief contributor to the S2 Geometry Library.
Re:How did Google get so big? by pixelpoet on Saturday July 28, 2018 @02:12AM (#17631835)
Thanks for this, anyone know what he's up to now?

I can't believe he left CG behind :(

Re:How did Google get so big? by wallflower on Friday July 27, 2018 @03:04PM (#17628331)
Thank you. I increased the answer to six words because you are very right.
Re:How did Google get so big? by ginger123 on Friday July 27, 2018 @04:17PM (#17628979)
I disagree with passion. The vision of Larry Page and Sergey Brin is responsible for Google's success. Jeff Dean, Sanjay Ghemawat executed Page and Brin's ideas, but they were not the key decision makers. Did Dean and Ghemawat decide to buy and then build Android? Did they decide to build Google Maps? Did they decide to buy YouTube?. The answer is no. Page and Brin would have hired other smart people if Dean and Ghemawat weren't available or had they left Google.
Re:How did Google get so big? by wallflower on Saturday July 28, 2018 @02:06PM (#17634234)
I agree, as I was not talking about the overall success of Google, just the success of Google's secret sauce (their backend infrastructure) that enabled it to sell ads and deliver search results with a high level of quality. If you read some of the books about early Google, they (Jeff, Sanjay et al) were really, really struggling to do stuff like index the web without the crawler collapsing mid-way. That's how MapReduce became to be developed.

Yes, Danger/HipTop, YouTube, and Keyhole are arguably the three smartest acquisitions that Google made.

Re:How did Google get so big? by snikeris on Friday July 27, 2018 @02:45PM (#17628187)
Also, the Google SRE book makes the point that they couldn't have achieved their scale w/out approaching ops from an engineering perspective. Linearly hiring ops people as services scale would have been too expensive.
Re:How did Google get so big? by 0xEFF on Friday July 27, 2018 @06:57PM (#17630147)
Approaching operations from a software development perspective wasn’t unique to Google.

Also ops is engineering and has been since before Google existed.

Re:How did Google get so big? by erikpukinskis on Saturday July 28, 2018 @02:53AM (#17631969)
Ops is probably more reasonably called engineering than most so-called “software engineering”.
Re:How did Google get so big? by y4mi on Saturday July 28, 2018 @03:11AM (#17632018)
Re:How did Google get so big? by jsty on Friday July 27, 2018 @02:48PM (#17628209)
Someday Google's going to have to list the risk of Jeff Dean leaving on their 10-K ... (not entirely joking)
Re:How did Google get so big? by WilliamEdward on Friday July 27, 2018 @03:52PM (#17628768)
Companies actually do this behind the scenes. They list individual employees as financial assets.
Re:How did Google get so big? by otp124 on Friday July 27, 2018 @04:14PM (#17628957)
Hmm, I hadn't heard of this practice before, and so I went searching and found this article to be a decent summary (posting here if others are curious): https://smallbusiness.chron.com/employees-intangible-assets-...
Re:How did Google get so big? by lloydde on Friday July 27, 2018 @07:42PM (#17630420)
Even some small companies (100+ employee) carefully track attribution of their top N% of non-VP employees and tie it into executive bonus.
Re:How did Google get so big? by dsnuh on Friday July 27, 2018 @07:18PM (#17630271)
Re:How did Google get so big? by Liron on Friday July 27, 2018 @07:26PM (#17630323)
No, there's no point for Google to pay a third party to insure any risk of $1B or less.
Re:How did Google get so big? by erikpukinskis on Saturday July 28, 2018 @01:40PM (#17634131)
Are there any risks at all worth insuring for Google? What’s the point of insurance if you have $100B in the bank?
Re:How did Google get so big? by ebikelaw on Friday July 27, 2018 @03:30PM (#17628578)
That doesn't provide a perfect answer. Sanjay and Jeff (and a lot of other senior Googlers) were all at DEC. Why didn't DEC succeed to the degree that Google has?
Re:How did Google get so big? by dunpeal on Friday July 27, 2018 @10:44PM (#17631217)
And Softdisk had John Carmack and basically the entire original Id software team on their payroll. And Fairchild Semiconductors had the entire executive founding team of Intel, and Xeroc PARC had not only several key people but essentially the entire IP portfolio of modern computers, including some working prototypes...

"Success" is the result of a very large equation with many variables, and having (some of) the right people is only a few of them.

Re:How did Google get so big? by dilyevsky on Friday July 27, 2018 @03:37PM (#17628648)
Because they didn’t have Craig Silverstein and many many others. Lets not pretend Jeff and Sanjay, as much talented and prolific as they are, are singlehandedly responsible for Googles success.
Re:How did Google get so big? by jasode on Friday July 27, 2018 @03:41PM (#17628670)
I don't know if Sanjay and Jeff worked directly on DEC's AltaVista search engine but that project was treated as a low-priority "demo showcase" for DEC Alpha servers instead of being seen by DEC management as a new multibillion dollar internet business to pursue. DEC could have had a "Bezos-hey-we-can-sell-computing-as-AWS" moment with AltaVista but they missed it and let Google surpass it.
Re:How did Google get so big? by nostrademons on Friday July 27, 2018 @04:19PM (#17628997)
They didn't - AltaVista was done by Louis Monier and Mike Burrows, both of whom also worked at Google and both of whom also made significant contributions to the codebase.

I do agree that management was the critical ingredient - my former VP at Google (employee #48) credited Urs Hoelzle (employee #9, and Google's first VP) as a key ingredient to Google's success. Ironically, though, Google hired Western Digital's management as well - SVP Alan Eustace came from there - and yet the team under Eustace at Google was much more effective than the team under Eustace at Western Digital.

I suspect that management has sort of a "gatekeeping" effect - if all of your management is good, the whole department does great things, but if a single link in the org chart is a dufus, the whole organization will fail. When I joined Google in 2009, my whole management chain (6 levels, up to Eric Schmidt) were all engineers, and it showed.

Re:How did Google get so big? by pm90 on Friday July 27, 2018 @04:58PM (#17629347)
> I suspect that management has sort of a "gatekeeping" effect - if all of your management is good, the whole department does great things, but if a single link in the org chart is a dufus, the whole organization will fail. When I joined Google in 2009, my whole management chain (6 levels, up to Eric Schmidt) were all engineers, and it showed.

So true. As an engineer, the most direct impact on my productivity has been the team and manager. But the manager is a SPOF: if he/she is bad, then my productivity suffers.

I have yet to work in an organization where there was a good chain as you mention though. Maybe that's what sets Google apart. Smaller companies tend to have shorter management chains and I prefer them for that reason.

Re:How did Google get so big? by foobarian on Friday July 27, 2018 @08:23PM (#17630663)
Mike Burrows was a coauthor on the algorithm behind bz2 [1]. It's the most mindblowing algorithm I ever encountered and so I have utmost respect for that couple of guys :)

[1] https://en.wikipedia.org/wiki/Burrows%E2%80%93Wheeler_transf...

Re:How did Google get so big? by kofejnik on Friday July 27, 2018 @09:18PM (#17630883)
thanks for sharing, it seriously is amazing
Re:How did Google get so big? by stuckinarut on Friday July 27, 2018 @05:52PM (#17629724)
They also had Dave Cutler and loads of other high profile people over the years.

Dozens have companies have benefited from the diaspora of DEC, and now Sun.

Re:How did Google get so big? by rhymenoceros on Friday July 27, 2018 @10:38PM (#17631195)
> To paraphrase some Cal/UC Berkeley professor that I cannot remember, he said that the problem with scaling is that when you go up a magnitude or more, you may need a qualitatively different solution for the same problem.

You might actually be thinking of Jeff's Stanford talk from a few years ago (2011). Slide 21 of:

https://static.googleusercontent.com/media/research.google.c...

  Ensure your design works if scale changes by 10X or 20X
    but the right solution for X often not optimal for 100X
There's no attribution, but that is good advice I've kept with me throughout my career. Props to the original author (perhaps apocryphal at this point?).
Re:How did Google get so big? by dmead on Friday July 27, 2018 @09:28PM (#17630933)
"And now their team's attention is on AI..." is quite the hype thing to say. Just because you can build a bigger bridge doesn't mean you're a good car designer.
Re:How did Google get so big? by nl on Saturday July 28, 2018 @02:29AM (#17631884)
Dean's PhD was in neural networks.

Also TPU2 is the only decent alternative to NVidia hardware in production, so that's a decent sign.

Re:How did Google get so big? by mrep on Friday July 27, 2018 @11:56PM (#17631428)
> "And now their team's attention is on AI..." is quite the hype thing to say.

Googles AI blows every other companies out of the water.

> Just because you can build a bigger bridge doesn't mean you're a good car designer.

What? How is that analogous?

Re:How did Google get so big? by paganel on Saturday July 28, 2018 @06:15AM (#17632479)
> Googles AI blows every other companies out of the water.

That is certainly true, but that could also mean that Google will be the first company to hit certain unsurpassable road-blocks. Just look at Google Translate, which IMO hasn’t qualitatively improved for the better for at least 2 years now, even though most probably the data on which it is based is at least an order of magnitude “bigger”. The same might also happen to autonomous driving.

Re:How did Google get so big? by fooker on Friday July 27, 2018 @11:39PM (#17631383)
They have proven they are a good 'car designer' too over the last 3 years.
Re:How did Google get so big? by grosjona on Friday July 27, 2018 @04:07PM (#17628895)
I think that Google's dominance has almost nothing to do with technology and almost everything to do with customer lock-in.

For me, duckduckgo.com is as good as Google but I still use Google because it's the default search engine for Chrome's address bar - And that's where I do all my searches from these days. If Duckduckgo made their own browser which was of the same quality as Chrome then I would definitely consider using it and switching to Duckduckgo for search as well.

Re:How did Google get so big? by magicalhippo on Friday July 27, 2018 @05:06PM (#17629414)
For me it's because with Google I don't need to go to page 5 to find interesting results.

I've tried to use DuckDuckGo several times, each time I've quickly found myself trawling page after page of results, reverted to Google and found the stuff I was after on page one.

Re:How did Google get so big? by simias on Friday July 27, 2018 @05:54PM (#17629736)
That used to be true for me but I've used ddg as my main search engine for a couple of years and I find myself using "!g" less and less. Actually you made me realize that I haven't used it in a while, probably weeks (and I've had a very busy couple of weeks with tons of opportunities to look for stuff online).

It's not because duckduckgo finds everything I want, it's because I've noticed that when ddg doesn't find something usually google doesn't either. Also whenever I end up on google now (generally because I forgot to change the search engine after a fresh install) I find the website cluttered which is a shame because it used to be in Google's DNA to keep the search page simple.

For instance I just made a completely random search on Google just to see what it would look like: https://svkt.org/~simias/up/20180727-235325_tou.png

What the hell is all of that stuff? If I wanted to browse the stock exchange or google maps I would. "People also search for" well good for them I guess.

Re:How did Google get so big? by monkeydust on Friday July 27, 2018 @05:23PM (#17629525)
100% same I wanted to love ddg but it's just not as good
Re:How did Google get so big? by amelius on Saturday July 28, 2018 @12:06AM (#17631459)
If only they implemented "!g" as a simple button, I'd use DDG more often. Typing "!g" requires 6 touchscreen actions on mobile :/
Re:How did Google get so big? by golfer on Saturday July 28, 2018 @02:23AM (#17631863)
DDG doesn't have stock prices, movie listings, or any of the useful "onebox" type information that Google has. It's just not as useful IMHO.
Re:How did Google get so big? by yellowcherry on Friday July 27, 2018 @05:19PM (#17629502)
I have exactly the opposite experience, time after time after time.

The one area where Google search provides me with a notably better experience is in image search.

Re:How did Google get so big? by nostrademons on Friday July 27, 2018 @04:40PM (#17629163)
Could also just use Tor Browser - it's Firefox modified for privacy, all running through Tor, with search defaulted to DDG. Actually works pretty well as a drop-in system, and you can't beat the privacy.

Ironically, my biggest complaint is the search - a number of queries that I'd expect to "just work" on Google, DDG really struggles with. There actually is a noticeable difference in quality between them.

Re:How did Google get so big? by smileysteve on Friday July 27, 2018 @04:50PM (#17629268)
You know you can change where the browser bar searches; In chrome, in Safari, and in Firefox, and Edge.
Re:How did Google get so big? by zjaffee on Friday July 27, 2018 @06:10PM (#17629852)
There is a huge benefit to being the default, if that wasn't the case google wouldn't pay all the companies they do billions of dollars a year to be the default.
Re:How did Google get so big? by dredmorbius on Friday July 27, 2018 @07:03PM (#17630181)
That wasn't true until recently on Chrome/Android, and seems to be a direct response to EU actions.
Re:How did Google get so big? by tonysdg on Friday July 27, 2018 @04:17PM (#17628987)
Out of curiosity, what prevents you from using Firefox? It allows you to specify a default search engine -- which you can set directly to DDG.
Re:How did Google get so big? by hk__2 on Friday July 27, 2018 @04:50PM (#17629266)
> Out of curiosity, what prevents you from using Firefox? It allows you to specify a default search engine -- which you can set directly to DDG.

Chrome allows you as well.

Re:How did Google get so big? by dcow on Friday July 27, 2018 @04:20PM (#17629003)
I use FF and DuckDuckGo. It's super easy.
Re:How did Google get so big? by arminiusreturns on Friday July 27, 2018 @04:51PM (#17629277)
I also heavily use the cli ddg search. Might try that out for anyone else who lives in the terminal.
Re:How did Google get so big? by tEMporality7 on Friday July 27, 2018 @04:37PM (#17629140)
I used to wonder why search engine companies care about which engine is the default in a browser because it only takes a few seconds to change it but now I know why. People are lazy :/
Re:How did Google get so big? by SllX on Saturday July 28, 2018 @01:52AM (#17631770)
It gets worse. I remember I had a friend that had a piece of spyware that set his default search engine to something else. At first glance I didn't realize it was a piece of spyware because all his searches looked like normal searches, albeit on a generic looking page.

I wondered why he didn't switch to Google or something else at the time and it used to annoy me whenever I was using his desktop for anything. Then I took a look into it and found out that he actually couldn't set his default search engine through his browser anymore, in Firefox or Chrome actually.

It was a browser extension. For that entire time there was a piece of spyware on his computer probably watching all of his browsing that he probably could have figured out about himself, if he wasn't too lazy to just switch his search engine. Once I looked into it, it took all of five minutes (if that) to figure out what was happening and remove the problem, and he's not exactly tech illiterate. He literally just didn't care or want to take the time to switch his search engine.

That's a data point. Just one, anecdotal data point, but take it how you will.

Re:How did Google get so big? by still_grokking on Friday July 27, 2018 @07:22PM (#17630297)
Re:How did Google get so big? by eksemplar on Friday July 27, 2018 @06:04PM (#17629805)
DuckDuckGo is terrible for non English speakers though.
Re:How did Google get so big? by cma on Friday July 27, 2018 @05:09PM (#17629431)
If that happened Google would probably ban Android phones that use Google apps from using third party browsers (like they already did for third party search providers). Then your tabs wouldn't be able to sync with mobile.
Re:How did Google get so big? by vongomben on Friday July 27, 2018 @06:16PM (#17629905)
Or you can use Firefox