> The boards using their chips need to fit into commodity interfaces (PCIE? DIMM? ) in Open Compute hardware

I mean, it's not exactly like pcie interfaces are ultra hard, must be done in house, cutting edge technology anymore. And why the huge emphasis on OCP? Nothing there necessitates any kind of extreme innovation.

> Their IP needs to actually be defensible. Otherwise, an Intel/Samsung whose manufacturing prowess & channel reach is multiple times that of Gorq will undercut the pricing with almost the same performance/watt. Oh, and they'll happily play nice with Open Compute, other standards bodies.

This has been true historically, but TSMC currently can out manufacture both intel and Samsung wrt 7nm. In addition, the steamroller of Moore's law no longer really holds with dennard scaling dead and wire/MOL rc and variation soaking up performance gains.

Btw, most of these criticisms could have been levied against nvidia way back when.

Not a criticism, merely truths about the industry.

Also, from Nvidia's founding to now, things have radically changed. It's depressing how winner-take-all semi has become. Nvidia could tape-out at $300k, today it's $2M. Number of semi players has reduced drastically even within the last 5 years

Nvidia didn't succeed because of their hardware manufacturing. They succeeded because they were dirty and ruthless on the market, and to this day will buy or sue the living fuck out of anything that could threaten their dominance.

If you look at the spaces where ASICs are still successful, one thing that is clearly important that you don't mention is the software kit around the chip(s). Half the team (or more) will be software guys. It's not about fabbing (since these companies are fabless and typically TSMC), it's about software support.

This is a great point; AMD has had chips for quite a while, but only just announced a TensorFlow release that can use them, and it's a binary release, rather than them successfully upstreaming their changes.

they can be just acquired back to Google or by another player for about $10M/head.

yep this has google acquisition/aquiihire written all over it IMO

The circle of life Google to startup back to Google via acquisition

For what? Acquisition only makes sense if a market opportunity is so close that missing it will affect the bottom line. I believe companies can independently build their own TPUs at lesser than $60M within 2 years. And even then the volumes for these chips are ... ?

Unless millions of units are being sold/month, it might just be cheaper to have some AI acceleration done in Intel's integrated CPU+FPGA offerings [1].

[1]: https://www.nextplatform.com/2018/05/24/a-peek-inside-that-i...

>I believe companies can independently build their own TPUs at lesser than $60M within 2 years.

I think we have very different work history wrt. the size/agility/cost of doing things/etc. of the companies that we've worked at :)

50 employees * $250,000/annum * 2 years = $25M. At 3 years, it's $37M. Realistically, for $35M you should have a working, reliable TPU if you're already a semi company, like Samsung

>$250,000/annum

it is bare minimum that you'd have to pay to an engineer, and i'm still not sure that you can find somebody able to do hardware design in AI space on that kind of low money today.

Also - the actual cost of an employee is significantly higher than what you pay to the employee. Plus everything else you need to run R&D company, especially if it involves hardware prototyping, etc.

I mean i can believe that a hot startup where people would take equity and see good chances at good exit can probably reach some reasonable prototype (not actual product) stage on $60M. For any bigger established company - it would be several times more expensive with an order of magnitude higher chances to screw the project (due to incorrect product fit, internal politics, overall internal inefficiency, managers actually hiring a crowd of "AI hardware designers" at $250K :), etc.)

Look at Cisco - the company does understand that they can't do anything new in house, so they regularly spin-off a $100M wad of cash (with usually the same 3 guys attached) and buy it back 1-2 years later for $700-800M. If Cisco attempted to invest those $800M into new tech development in-house the chances for a good outcome would be much lower.

> >$250,000/annum

> it is bare minimum that you'd have to pay to an engineer, and i'm still not sure that you can find somebody able to do hardware design in AI space on that kind of low money today.

Maybe in SV. In Dallas, only top analog and digital designers are commanding that kind of salary. There's plenty of fab and board shop capacity there, too.

Also, I don't see why you specifically need "somebody able to do hardware design in [the] AI space". Experienced digital and mixed-signal designers should be sufficient.

Remember, the cost to the employer is roughly double the employee's salary, therefore the $250,000 per annum cost means the employee salary is $125,000 per annum. Not exactly top dollar.

Where did you get an 100% overhead for an employer?

I've done back-of-the-envelope estimates before and have come nowhere near that. At best, it's somewhere between 15-25% depending on employer 401(k)/etc matching, deferred comp, and office space pricing.

The worst I could find is 40% - http://web.mit.edu/e-club/hadzima/how-much-does-an-employee-... - and that uses a much different cost basis (not $125k).

Where in the world did you come up with that number?

Sure, everyone has a different number. For example, in the article you linked to, there is this metric: "the fully functioning managed employee costs about 2.7 times the base salary".

The person I was replying to specifically said "pay to an engineer". That is not the fully-loaded cost.

The person he was replying to was talking about cost to design the chip, which would imply employer costs. The error has to stop somewhere.

I think a lot of next generation chips are not just squeezing more compute into hardware, but examining whether we can change the nature of neural net compute ro better fit what we can do in hardware. That's Graphcore's pitch at least. And plenty of people are looking into whether you can do non-IEEE754 math in a way that works for compute, e.g. FP16, quantized arithmetic, etc.

You probably don't need the entire team to be familiar with the math, but someone has to be.

And as other comments have noted, Software is actually going to be a very big deal.

No one gets paid 250K because they know what fp16 minifloats are

At some point it wasn't widely known that you could do NN training with fp16. Now it's common knowledge, but what is the next thing?

Hardware design with synthesis is much easier than you think and costs a lot less.

I'd add one more: they need to be able to ship new product at the cadence of Nvidia, Intel et al. Showing wins in the lab is very different from getting that product to market before the next round of improvements from competitors overtakes you.

And then you need to do it again.

I think this was more true back when Intel was in the thick of the slope of Moore’s law.

If new accelerators slip into existing hardware and don’t increase the power budget, one could get millions of installs in old and existing hardware.

If this is a beefy hot accelerator, that is a totally different story.

Maybe. I am not saying you're wrong, but we're dealing with orders of magnitude Innovation, and Clayton M. Christensen has shown us that when it comes to disruptive tech, the status-quo standard and incumbents are not the yardstick that they once were.

The Future is not what it used to be.

I stand by my "Maybe", 100% guaranteed.

- The boards using their chips need to fit into commodity interfaces

USB? There are USB GPU boxes too.

USB? USB 3.1 is 16X slower than PCIE and maybe 100X slower than DIMM. USB is not an OCP recommended peripherals interface, it's for consumer removable products.

Then again, I'm not entirely sure about the requirement for a high speed interface in AI. I'm assuming a fast interface is needed as the amount of data ingested is typically large

People use DIMM as a general purpose interface?

No, but it can be done. I worked on an FPGA board interfacing through the DRAM interface once: https://www.researchgate.net/publication/4139277_Pilchard_-_...