Ask HN: Why can't Nvidia compete in the inference market?

Posted by shegerking2020 on Saturday April 27, 2024 @07:59PM
from the ask-hn dept.

Hey HN! Lately I have been thinking about how the inference market will involve. We saw that llama 3 with groq was showing great results. Once question I had was why can't nvidia compete in the inference market? In the video below chamath was saying how nvida won't be able to compete in inference. https://youtu.be/1ZQ33OnGFWE?t=1254 is here just chilling his bag or are there some truth to this? What are fundamental reason that this would be the case?

( 2 comments | on HN )

Re:Ask HN: Why can't Nvidia compete in the inference market? by talldayo on Saturday April 27, 2024 @08:07PM (#40184747)

Likely because they don't make inference accelerators, they make GPUs. It's entirely plausible that someone could optimize the hardware pipeline better by incorporating less complex logic; I wouldn't expect Nvidia to try that though, not with their main products.

[ Parent | on HN ]

Re:Ask HN: Why can't Nvidia compete in the inference market? by st380752143 on Saturday April 27, 2024 @09:47PM (#40185306)

This reminds me of a sentence Jensen said internally: "We do the thing that only we can do"

[ Parent | on HN ]