Yeah, it's not even like they're running the datacenters where the training and tuning are happening. I would hope some of the people understand what current compute requirements are and perhaps they know better than most what future requirements will be. However, MS has been doing most of the backend for OpenAI and they've been in discussions with actual silicon architecture people (not just NVidia), but those are the folks who would do any implementation.

Perhaps they'll pull off an Apple (for ARM) and do their own architecture (either for training/tuning or inference) that will have a significant effect on the industry, but it seems unlikely. They haven't hired the right people.

The real advantage they might have is insight into how the algorithms can be adapted to reduce power consumption/latency while improving performance. It would seem odd to me, if there weren't more than an order of magnitude in new algorithms for LLMs. You're not going to get 10x the transistors or speed from silicon, but you might get an efficient architecture for a significant algorithmic improvement (that might not just be CUDA).

"I know how machine learning and statistical computing works, therefore I am an expert in hardware design" fallacy.

I am guessing an incredibly talented team that is incredibly networked and incredibly well funded and proven agile in the tech hub of the world can find hardware experts. Don’t know why anyone would bet against that.

We would have heard if they had hired/bought the size of team necessary to design a system large enough to be a significant impact. Modern (eve sub 28nm much less 2nm) design is hugely complex and the range of things that an AI compute engine needs to do are very broad.

Perhaps they could design a core and license it out? I'm trying to come up with a way they can do something significant without 100 people. Just the memory and serial connections are complex enough ignoring the GPU or heat/power issues.

It took apple like 10 years to go from their first chips to actually using them in laptops, and they are literally the most well capitalized company on the planet. Sorry if I'm skeptical that some relative up starts with a billion in compute from Microsoft can compete with trillion dollar companies that have been around for decades.

I wouldn't bet against it but that approach has a remarkably low rate of success. We hear about the winners - survivorship bias is real.

Nobody can even define what AI is, why we need it, or how to achieve it. Usually it makes sense to seek funding to execute on a plan. Making a fancy chat bot that scrapes the web to synthesize sometimes accurate and sometimes useful information is not worth trillions of dollars.

What is essentially happening in my opinion is technical innovation has slowed so silicon valley is seeking money to prop up a house of cards that doesn't make much new that is useful or needed.

Can anyone specifically say what trillions of dollars invested in "AI" would buy for society?

It seems to me there are so many higher priorities.

> "I know how machine learning and statistical computing works, therefore I am an expert in hardware design" fallacy.

A typical case of engineer's disease.

Said the horse factory when automobiles were being built.

I don't remember LLM's claiming to replace GPU's. This is more like arguing with a landowner why your assembly line is so innovative and needs to be built on their land for free. They need the land, the land doesn't necessarily need them yet.

Absolutely terrible analogy.

A LLM might "believe" that horses are built in factories.

Pullman Company will disagree with you.

How about something along the lines of AWS and their Graviton?

Graviton - you mean the poorly performing solution that only has a space in the market because amazon sells it as a subsidized cost as part of a larger effort to put pricing pressure on amd/intel? That Graviton?

Was Google a chip designer before the first TPU?

Yes. Google had a number of chip products before that. Some made it to A1 and worked. Just cause they don’t advertise it doesn’t make it not so.

> Yes. Google had a number of chip products before that.

Is that true? I can't find anything suggesting it is. In fact, the little I can find suggests you are incorrect. I'll link them for the sake of referencing sources but they're both pretty awful ad-ridden sites...

A 2016 Tech Radar interview [0] with Norm Jouppi has him quoted as saying:

> [The] Tensor Processing Unit (TPU) is our first custom accelerator ASIC [application-specific integrated circuit] for machine learning [ML], and it fits in the same footprint as a hard drive.

And a 2023 Tom's hardware post [1] begins:

> Google has made significant progress in its endeavor to develop its own data center chips, according to a new report. The Information says that a key milestone has just been reached, which means that Google can plan to roll out server systems powered by the new chips starting from 2025.This is not the first processor that Google has successfully put through R&D - the company has previously made an ASIC for servers and an SoC for mobile devices. The search giant started using its internally developed Tensor Processing Unit (TPU) as far back as 2015.

[0]: https://www.techradar.com/news/computing-components/processo...

[1]: https://www.tomshardware.com/news/google-reaches-self-develo...

I guess it depends on what you are defining as a chip and what you are defining as "Google" -- as in if they have contractors design/build to their needs does that count.

1/ https://www.wired.com/2012/03/google-microsoft-network-gear/

2/ I believe they had a few custom chips designed for the youtube workloads that predate the TPU.

I remember in 2010 there was a building in MV that focused on custom chips.