News for hackers. Stuff that matters.
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Wednesday November 30, 2022 @03:24PM (#33807102)
I found graviton to be a mixed bag. It was certainly extremely fast when using the very high end instances and I tested it successfully using a Rust based message queue system I was writing and it got some ridiculously fast number like 8 million messages a second, from memory, using the fastest possible graviton instance (this was about 18 months ago).

I did try to switch some of my database servers to it a couple of years ago and after random hangs, I gave up and went back to intel. I tried again further down the track and same thing - random hangs. I assume this sort of thing comes with a new architecture but I'd be hesitant to move any production infrastructure to it without extensive long term testing.

In the case of graviton based GPU instances I found that the GPU enabled software I wanted to use didn't work.

If you are comparing performance, I'd suggest buying a fast AMD machine and run it locally and compare performance - local servers tend to be much faster and cheaper than cloud. And if your application uses GPUs then if you possibly can then its very much in your interests to run local servers.

Re:Switching to AWS Graviton slashed our infrastructure bill... by Dowwie on Wednesday November 30, 2022 @03:40PM (#33807326)
I'm curious about that Rust-based message queue system
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Wednesday November 30, 2022 @03:51PM (#33807473)
What do you want to know? It was a prototype. I was trying to learn Rust (didn't succeed), but I did manage to hack together a message queue that used HTTP for client interaction.

I'd previously written a SQL database message queue in Python which worked with Postgres/MySQL and SQL server. This worked well but it was not fast enough for my liking. My goal was to build the fastest and simplest message queue server that exists, with zero configuration (I hate configuration).

I used Rust with Actix and I tried two strategies - one strategy was to use the plain old file system as a storage back end, with each message in a single file. This was so fast that it easily maxed out the capability of the disk way before the CPU capabilities were maxed out. The advantage of using plain old file system as a storage back end is it requires no configuration at all. So I moved on to a RAM only strategy in which the message queue was entirely ephemeral, leaving the responsibility for message persistence/storage/reliability to the client. This was the configuration that got about 8 million messages a second.

As far as I could tell my prototype left almost all message queue servers in the dust. This is because message queue servers seem to almost all integrate "reliable" message storage - that makes the entire solution much, much more complex and slow. My thinking was to separate the concerns of storage/reliability/delivery and focus my message queue only on message delivery, and push status information back to the client, which could then decide what to do about storage and retries.

I gave up because I didn't see the point in the end because it wasn't going to make me any money, and I was finding Rust frustratingly hard to learn and I had other things to do.

Re:Switching to AWS Graviton slashed our infrastructure bill... by moloch-hai on Wednesday November 30, 2022 @05:39PM (#33808982)
It seems very diplomatic of you to say you found Rust hard to learn, rather than that it was hard to make Rust do what you wanted. You seem very clear on what you wanted to do.
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Wednesday November 30, 2022 @06:04PM (#33809288)
I managed to build it without really grasping Rust by hacking around and looking at examples of how other stuff worked that I wanted to do, and by avoiding doing things in Rust that I didn't understand - stuff as basic as function calls.

The resulting code worked but was garbage and at the end of the day Rust had not clicked for me and being fluent in it still felt a distant goal.

I love the idea of Rust but I don't like the implementation.

I'm hoping in time there will be a new language created that has the memory and thread safety of Rust but is 50 times more simple.

Re:Switching to AWS Graviton slashed our infrastructure bill... by estebank on Wednesday November 30, 2022 @11:12PM (#33812235)
If you don't mind, what where the things that were hard to deal with? You mention "function calls", but I'd like to understand what that actually means in practice. This kind of negative feedback is useful to improve Rust for any other newcomers, even if we have already soured you for good.
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Thursday December 01, 2022 @12:50AM (#33812844)
I could not even succeed in doing the simplest thing like putting code into a function and executing the function and getting a result back. I can't recall why.
Re:Switching to AWS Graviton slashed our infrastructure bill... by likeabbas on Thursday December 01, 2022 @07:38AM (#33815453)
Interestingly, I started a side project to rebuild OneTimeSecret in Rust yesterday and got the basics working already https://github.com/likeabbas/RustOneTimeSecret

The only other exposure I had to the language was attempting some Leetcode problems. I really like Rust so far. The type system is extremely powerful, the compiler gives robust error messages, and it has so many awesome features like pattern matching and destructuring.

Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Thursday December 01, 2022 @06:31PM (#33824596)
You are smarter than me.
Re:Switching to AWS Graviton slashed our infrastructure bill... by likeabbas on Friday December 02, 2022 @08:53AM (#33830402)
Doubtful. I may have wanted to like Rust more than you when giving it a shot.

I've written a lot of Java and Go but I never enjoyed writing in those languages. I found functional programming languages in college to be very fun, and Rust is giving me that same feeling.

Re:Switching to AWS Graviton slashed our infrastructure bill... by mathewvp on Thursday December 01, 2022 @01:17PM (#33820224)
> If you don't mind, what where the things that were hard to deal with?

The language syntax itself. Looks like the language developers wanted rust to stand out and purposefully went out of way to make it extremely difficult to learn. This is one area where Go shines, you can learn the syntax in half a day.

Re:Switching to AWS Graviton slashed our infrastructure bill... by moloch-hai on Thursday December 01, 2022 @01:51PM (#33820767)
Rust syntax shoulders a load of semantics most languages don't bother trying to represent. They could have made it more wordy, instead, but chose the much shorter symbols, many of which can often be ignored when skimming, more easily than keywords would have been.

They have not loaded up the language with any unnecessary syntax. Go lacks the syntax because it is unable to express the semantics, and is (deliberately) a much less capable language for that. Rust is meant for professionals, so must be able to express whatever they may need to say, where Go was meant to keep junior coders out of trouble.

Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Thursday December 01, 2022 @06:31PM (#33824594)
>> They have not loaded up the language with any unnecessary syntax.

Rust is composed of at least six sub languages.

Re:Switching to AWS Graviton slashed our infrastructure bill... by estebank on Thursday December 01, 2022 @06:44PM (#33824760)
What syntax would you remove?
Re:Switching to AWS Graviton slashed our infrastructure bill... by mathewvp on Friday December 02, 2022 @01:12AM (#33827397)
> Rust is meant for professionals, ..., where Go was meant to keep junior coders out of trouble.

So we the not so elite coders will stick with Go, thanks.

Re:Switching to AWS Graviton slashed our infrastructure bill... by likeabbas on Wednesday November 30, 2022 @06:59PM (#33809967)
fluvio.io ?
Re:Switching to AWS Graviton slashed our infrastructure bill... by neodypsis on Wednesday November 30, 2022 @03:25PM (#33807119)
Which database are you using?
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Wednesday November 30, 2022 @03:26PM (#33807137)
pg
Re:Switching to AWS Graviton slashed our infrastructure bill... by neodypsis on Wednesday November 30, 2022 @05:48PM (#33809101)
Was this a while ago or was it a recent experience? I'm asking because I'm planning on using a serverless instance of PG and was interested in trying the ARM64 version.
Re:Switching to AWS Graviton slashed our infrastructure bill... by andrewstuart on Wednesday November 30, 2022 @06:00PM (#33809247)
About 18 months ago.

Try it - it might work fine for you.

Re:Switching to AWS Graviton slashed our infrastructure bill... by jdmichal on Wednesday November 30, 2022 @08:32PM (#33811041)
We've been running RDS PostgreSQL on r6g instances for the past few months with no issues.
Re:Switching to AWS Graviton slashed our infrastructure bill... by glogla on Wednesday November 30, 2022 @03:48PM (#33807441)
There are huge differences in the machine generations. We found that for our workload Graviton3 (c7g) is the best, followed by AMD (m6a), followed by Intel (m6i) with Graviton2 (m6g) somewhat lagging. We can't use Graviton3 however because of memory limitations, so we're using AMD. The difference to the old machine types (m5) is staggering, the m6a is basically twice the performance of m5, while being cheaper.

However, I've seen a lot of benchmarks telling a different story, so it is important to actually measure your workloads.

Re:Switching to AWS Graviton slashed our infrastructure bill... by whalesalad on Wednesday November 30, 2022 @03:48PM (#33807447)
I’ve been enjoying them here and there but I’ve also found that for some of my workloads a high clock Intel node is required. Even the Epyc nodes couldn’t keep up. I don’t completely know why, never dug too far into it.
Re:Switching to AWS Graviton slashed our infrastructure bill... by judge2020 on Wednesday November 30, 2022 @03:54PM (#33807519)
> local servers tend to be much faster and cheaper than cloud.

Of course, running a server in your house is not going to achieve five or even three 9's of reliability, and even colocating a single rack in a single location might be more expensive than putting that infra in AWS (depending on how data-heavy your use case is, given AWS' exorbitant data transfer costs).

Re:Switching to AWS Graviton slashed our infrastructure bill... by lantry on Wednesday November 30, 2022 @04:32PM (#33808052)
you can hit three nines even if you're down for 1.5 minutes every day, or ten minutes a week. It's really not as hard to hit as it sounds. For a compute heavy process that isn't end user facing (e.g. batch processing) it's perfectly viable.

https://uptime.is/

Also, most cloud providers don't guarantee five nines anyway. GCE SLA is 99.5 on a single instance, 99.99 on a region

https://cloud.google.com/compute/sla

Re:Switching to AWS Graviton slashed our infrastructure bill... by no_wizard on Wednesday November 30, 2022 @04:03PM (#33807643)
I'd argue just find a different cloud provider.

GCP, Azure, Supabase, Cloudflare etc if you want managed services.

If you want a mix of managed services and raw compute, look more at Fly.io, Linode, Digital Ocean perhaps?

I have found AWS being the "cheapest" or even "reasonable" in the cost department to be slimmer every year.

Re:Switching to AWS Graviton slashed our infrastructure bill... by MuffinFlavored on Wednesday November 30, 2022 @04:22PM (#33807906)
Cloudflare doesn’t let you host Docker containers or offer managed Postgres do they?
Re:Switching to AWS Graviton slashed our infrastructure bill... by no_wizard on Wednesday November 30, 2022 @04:33PM (#33808060)
Its all about how they may fit in your stack. Most definitely fly.io does. I think Cloudflare as far as I'm aware is they're looking at supporting Docker.

I just listed managed services (not all of them may fit I imagined)

Re:Switching to AWS Graviton slashed our infrastructure bill... by jiggawatts on Wednesday November 30, 2022 @06:23PM (#33809512)
Steer clear of Digital Ocean.

They've had senior staff on HN justifying security lapses that commenters were describing as a "clownshoes operation".

Re:Switching to AWS Graviton slashed our infrastructure bill... by axiak on Wednesday November 30, 2022 @04:38PM (#33808134)
Arm has a much looser memory model than x86 [1 for a comparison]. It's possible that the random hangs are due to a race condition in PG that doesn't show up in x86 because memory visibility doesn't require as much synchronization.

1: https://www.nickwilcox.com/blog/arm_vs_x86_memory_model/