News for hackers. Stuff that matters.
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by andrewstuart on Thursday April 13, 2023 @04:12PM (#35561456)
I used to be an AWS true believer. Now I find it increasingly difficult to be enthused about anything AWS. It’s all so expensive, complex and locked in. I recently shut down everything I had on AWS except Route 53 and workmail, both of which I really like. The craziest AWS thing is that they moved all GPU instances to a quota system where you have to request access, specifying the number and type and location of instances that you want to run. It’s like Azure which has an equally terrible quota system. Anyhow I needed to run some performance tests in GPU instances. In the past I would have spun up a bunch of different instance types, run my tests and moved on. In this case I applied for a quota of one machine type and 24 hours later AWS got back to me to approve my request to run one instance. At that point I gave up on the idea of AWS being the heart of any GPU based infrastructure.
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by mochomocha on Thursday April 13, 2023 @04:38PM (#35561733)
They behave this way because they are so tight on GPU capacity given incredible demand. Getting a GPU instance on GCP is as miserable of an experience.

Try Lambda Labs, there's usually more availability, and more importantly you get some visibility into currently available capacity.

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by ceejayoz on Thursday April 13, 2023 @05:10PM (#35562105)
GPU instances are also a favorite option for crypto miners using compromised AWS accounts. They run up $20k in bills in a short period, which AWS typically nulls out if you beg.
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by crazygringo on Thursday April 13, 2023 @04:35PM (#35561691)
Everything you say seems equally applicable to any cloud provider.

They're all expensive and they all want to lock you in, because they're businesses. GPU's are in extremely high demand now so that's just how it is. And they're complex because different customers have different needs, so that's inherent.

I don't see how any of your complaints are anything a for-profit cloud provider could change and still survive as a business.

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by dubcanada on Thursday April 13, 2023 @04:32PM (#35561658)
They all are, what did you switch too? Suppose it's kind of the point really, it's a cloud with a bunch of "tools".

You are welcome to install MySQL or elasticsearch on another VPS outside of AWS and use it. But that's not what customers want.

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by qbasic_forever on Thursday April 13, 2023 @04:14PM (#35561466)
It's not like "Open" AI is any less locked in or opaque though...
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by verdverm on Thursday April 13, 2023 @04:25PM (#35561596)
OpenAI is a very limited API surface and there are already projects out there creating interfaces on top of N GPT implementations spanning private & public, giving you choice.

The comparison here is apples & oranges

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by metadat on Thursday April 13, 2023 @04:42PM (#35561780)
I know Character.ai prefers Oracle Cloud for GPU compute workloads because OCI can actually deliver significant capacity in a reasonable amount of time (often < 24 hours, almost always within a few days for hundreds [or more] of GPU instances, especially if you aren't picky about the region).

I'm all for bagging in Oracle in general, yet it's undeniable this is one area where they're leagues ahead and winning.

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by skybrian on Thursday April 13, 2023 @05:24PM (#35562253)
It sounds like they temporarily have excess capacity? I don't think you can count on that to last.
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by asah on Thursday April 13, 2023 @05:22PM (#35562226)
is that because they are 10x the scale... or because they have 1/10th the customers demand...
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by SanderNL on Thursday April 13, 2023 @05:42PM (#35562490)
We told you this would happen, but no, we were luddites.
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by ayhoung on Thursday April 13, 2023 @10:46PM (#35564953)
besides GPU instances, have you tried out lightsail? i find its quite competitive with DO, Linode, etc. with AWS infra backing
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by cavisne on Thursday April 13, 2023 @11:24PM (#35565178)
GPU's in cloud is just a bad business unfortunately.

There is only one vendor (for anything scientific/ML anyway) and you have to buy specific (overpriced) models from them to be compliant with the license. So zero negotiating power, and everything is proprietary so you cant build any software/support edge.

You cant oversell a GPU like you can CPU/memory, and most of the computation happens on the chip so no one cares how good the rest of your stack is.

And finally you are in a neverending battle with crypto miners, which would never be profitable on cloud so they have no intention of paying you.

Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by lovesdogsnsnow on Friday April 14, 2023 @05:00PM (#35574702)
Aren't interconnect bandwidth and latency important for training workloads?
Re:Amazon announces 'Bedrock,' its ChatGPT and DALL-E rival by cavisne on Monday April 17, 2023 @01:09AM (#35597040)
Yes, but to move the needle you need to use another single vendor Mellanox (which nvidia owns).

So the (slight) differentiations are.

AWS - gpudirect and high bandwidth (but also high latency)

GCP - 16 gpus on one machine

Azure/OCI - mellanox