Chiplet ASIC supercomputers for LLMs like GPT-4

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by bhouston on Wednesday July 12, 2023 @05:54AM (#36692329)

The $0.87/hour price you gave is theoretical and also we know any price in a paper for compute is wrong by the time of publication.

Pragmatically the prices are closer to $2/hr according to this recent post here on Hacker News: https://llm-utils.org/Nvidia+H100+and+A100+GPUs+-+comparing+...

Although again prices change on a daily basis on spot providers.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by jacquesm on Wednesday July 12, 2023 @06:06AM (#36692413)

> The $0.87/hour price you gave is theoretical

https://cloud.google.com/blog/products/compute/a2-vms-with-n...

That's as close as I got to verifying that price.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by bhouston on Wednesday July 12, 2023 @06:21AM (#36692510)

Another list that shows pricing both constant and spot. The best GCP spot price is $1.1, but Jarvis seems to say its spot for the 40GB A100 is $0.69:

https://fullstackdeeplearning.com/cloud-gpus/

I feel there are more fair criticisms of that paper than its inclusion of the snapshot price of variable priced compute resource.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by jacquesm on Wednesday July 12, 2023 @10:10AM (#36694805)

Sure, to me it more of an extra item than the main one but it is one that you can readily verify because most of the other claims are far more vague. If they're willing to fudge on that one then I have much less confidence in the rest of their claims.

[ Parent | on HN ]

Re:Chiplet ASIC supercomputers for LLMs like GPT-4 by kraken12 on Wednesday July 12, 2023 @03:00PM (#36699997)

Seems like HN comments have determined that the cost number is not fudged..

[ Parent | on HN ]