News for hackers. Stuff that matters.

A comment on Who uses Google TPUs for inference in production?. Parent comment

Re:Who uses Google TPUs for inference in production? by mike_d on Monday March 11, 2024 @04:16PM (#39672771)
> I think the tech stack needs another 12-18 months to mature

Google has been doing AI before any other company even thought about it. They are on the 6th generation of TPU hardware.

I don't think there is any maturity issue, just an availability issue because they are all being used internally.

[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by htrp on Monday March 11, 2024 @04:24PM (#39672845)
100% agree, if I have access to the TPU team internally, it'll be very easy to use in production.

If you aren't internal, the documentation, support, and even just general bug fixing is impossible to get.

[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by ethbr1 on Monday March 11, 2024 @06:22PM (#39673902)
(Has an expert team dedicated solely to optimizing for exotic hardware) = an option

(Doesn't have a team like that) = stick to mass-use, commodity hardware

That's generally been the trade-off since ~1970. And usually, the performance isn't worth the people-salaries.

How many examples of successful hardware that isn't well-documented and doesn't have drop-in 1:1 SDK coverage vs (more popular solution) are there?

It seems like a heavy-lift to even get something that does have parity in those ways adopted, given you're fighting market inertia.

[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by chatmasta on Monday March 11, 2024 @07:50PM (#39674518)
Google sells access to TPUs in its cloud platform, so you'd think they would be more open about sharing development and tooling frameworks for TPUs. It's like Borg (closed source, never used outside Google, made them no profit) vs. Kubernetes (open source, used everywhere, makes them profit).
[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by fnord77 on Monday March 11, 2024 @11:29PM (#39675874)
> Google has been doing AI before any other company even thought about it

This not even remotely true. SRI was working on AI in various forms long before google existed

[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by TwentyPosts on Tuesday March 12, 2024 @03:53PM (#39684155)
Who or what is SRI?
[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by fluidcruft on Tuesday March 12, 2024 @11:25PM (#39687804)
See http://www.sri.com
[ Parent | on HN ]
Re:Who uses Google TPUs for inference in production? by fnord77 on Thursday March 14, 2024 @12:30PM (#39705985)
next to NASA, probably the most innovative organization in human history

https://www.sri.com/timeline-of-innovation/

[ Parent | on HN ]