Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival
by lovesdogsnsnow on Friday April 14, 2023 @05:00PM
(#35574702)
Aren't interconnect bandwidth and latency important for training workloads?
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival
by cavisne on Monday April 17, 2023 @01:09AM
(#35597040)
Yes, but to move the needle you need to use another single vendor Mellanox (which nvidia owns).
So the (slight) differentiations are.
AWS - gpudirect and high bandwidth (but also high latency)
GCP - 16 gpus on one machine
Azure/OCI - mellanox
There is only one vendor (for anything scientific/ML anyway) and you have to buy specific (overpriced) models from them to be compliant with the license. So zero negotiating power, and everything is proprietary so you cant build any software/support edge.
You cant oversell a GPU like you can CPU/memory, and most of the computation happens on the chip so no one cares how good the rest of your stack is.
And finally you are in a neverending battle with crypto miners, which would never be profitable on cloud so they have no intention of paying you.