Amazon is exactly the kind of company that would take 50% margin on their x86 servers and 0% margin on their Graviton servers in order to engineer a long term shift that's in their favor - the end of x86 server duopoly (or monopoly depending on how the wind is blowing).

Many people complain about AWS' networking costs, but I also suspect these are generally at-cost. A typical AWS region has terabits upon terabits of nano-second scale latency fiber ran between its AZs and out to the wider internet. Networking is an exponential problem; AWS doesn't overcharge for egress, they just simply haven't invested in building a solution that sacrifices quality for cost.

Amazon really does not have a history of throwing huge margins on raw compute resources. What Amazon does is build valuable software and services around those raw resources, then put huge margins on those products. EC2 and S3 are likely very close to 0% margin; but DynamoDB, EFS, Lambda, etc are much higher margin. I've found AWS Transfer for SFTP [1] to be the most egregious and actually exploitative example of this; it effectively puts an SFTP gateway in front of an S3 bucket, and they'll charge you $216/month + egress AND ingress at ~150% the standard egress rate for that benefit (SFTP layers additional egress charges on-top-of the standard AWS transfer rates).

[1] https://aws.amazon.com/aws-transfer-family/pricing/

Obviously it's on demand pricing and the hardware isn't quite the same, with Hetzner using individual servers with 1P chips. Amazon also has 10gbps networking.

But still, zero margin? Let's call it a factor of two to bridge the monthly and on demand gap - does it really cost Amazon five times as much to bring you each core of Zen 2 even with all their scale?

I don't think Amazon overcharges for what they provide, but I bet their gross margins even on the vanilla offerings are pretty good, as are those of Google Cloud and Azure.

Margin is a really weird statistic to calculate in the "cloud". Sure, you could just mortgage the cost of the silicon across N months and say "their margin is huge", but realistically AWS has far more complexity: the costs of the datacenter, the cost of being able to spin up one of these 32 core EPYC servers in any one of six availability zones within a region and get 0 cost terabit-scale networking between them, the cost of each of those availability zones not even being one building but being multiple near-located buildings, the cost of your instance storage not even being physically attached to the same hardware as your VM (can you imagine the complexity of this? that they have dedicated EBS machines and dedicated EC2 machines, and yet EBS still exhibits near-SSD like performance?), the cost of VPC and its tremedous capability to model basically any on-prem private network at "no cost" (but, there's always a cost); that's all what you're paying for when you pay for cores. Its the stuff that everyone uses, but its hard to quantify into just saying "jeeze an EPYC chip should be way cheaper than this"

And, again, if all you want is a 32 core EPYC server in your basement, then buy a 32 core EPYC server and put it in your basement. But, my suspicion is not that a 32 core EPYC server on AWS makes zero margin; its that, if the only service AWS ran was EC2, priced how it is today, they'd be making far less profit than when that calculation includes all of their managed services. EC2 is not the critical component of AWS's revenue model.

The marginal cost of VPC is basically 0. Otherwise they couldn't sell tiny ec2 instances. The only cost differences between t3.micro and their giant ec2 instances are (a) hardware and (b) power.

That's not strictly true. They could recoup costs on the more expensive EC2 instances.

I have not idea what the actual split is, but the existence of cheap instances doesn't mean much when Amazon has shown itself willing to be a loss-leader.

If they are recouping their costs, it's a capital expense, and works differently than a marginal cost. AWS's networking was extremely expensive to _build_ but it's not marginally more expensive to _operate_ for each new customer. Servers are relatively cheap to purchase, but as you add customers the cost increases with them.

If they're selling cheap instances are a marginal loss, that would be very surprising and go against everything I know about the costs of building out datacenters and networks.

I'm skeptical about this claim. Most cloud providers try to justify their exorbitant bandwidth costs by saying it's "premium", but can't provide any objective metrics on why it's better than low cost providers such as hetzner/ovh/scaleway. Moreover, even if it were more "premium", I suspect that most users won't notice the difference between aws's "premium" bandwidth and a low cost provider's cheap bandwidth. I think the real goal of aws's high bandwidth cost is to encourage lock-in. After all, even if azure is cheaper than aws by 10%, if it costs many times that for you to migrate all your over to azure, you'll stick with aws. Similarly, it encourages companies to go all-in into aws, because if all of your cloud is in aws, you don't need to pay bandwidth costs for shuffling data between your servers.

The reality is, their networks are fundamentally and significantly higher quality, which makes them far more expensive. But, maybe most people don't need higher quality networks, and should not be paying the AWS cost.

But the problem is that you can't. You simply can't use aws/azure/gcp with cheap bandwidth. If you want to use them at all, you have to use their "premium" bandwidth service.

What? My $2000 Titan V GPU and $10 raspberry pi both payed for themselves vs EC2 inside of a month.

Many of AWS's managed services burn egregious amounts of EC2, either by mandating an excessively large central control instance or by mandating one-instance-per-(small organizational unit). The SFTP example you list is completely typical. I've long assumed AWS had an incentive structure set up to make this happen.

"We're practically selling it at cost, honest!" sounds like sales talk.

*Unless things have changed in the last year or so since I looked

This seems to be demonstrably false, given that Amazon Lightsail exists. Along with some compute and storage resources:

$3.50 gets you 1TB egress ($0.0035/GB)

$5 gets you 2TB egress ($0.0025/GB)

Now, its certainly possible that Amazon is taking a loss on this product. Its also possible that they have data showing that these types of users don't use more than a few percent of their allocated egress. But I suspect that they are actually more than capable of turning a profit at those rates.

4$!

That's a theoretical maximum of 1Tb egress each three hours. So for the cost of 3 hours of egress you can buy an entire VPS with a month of egress, for cheaper. It's insane just how much cheaper it really is.

I assure you that you can run these servers for one hour a day and no one will bat an eye. I know people running seedboxes at full speed for 10 hours a day or so without an issue - that's 100 times the bandwidth of even Lightsail for the same price.

They have worse peering than AWS but the difference in cost to them is certainly not 100x or more.

Clearly they have margins, and fat ones at that.

"When factoring in heavy depreciation, AWS has EBITDA margins of around 50%."

https://seekingalpha.com/news/2453156-amazon-plus-15-percent...

Shameless plug - partly because of the high cost of sftp in AWS, and lack of ftp (understandable), and a bunch of people wanting the same in Azure / GCS, that made us start https://docevent.io which has proved quite popular.

And to make another point. Apple isn't a lower-end cost cutting laptop makers.

Apple sell ~20M Mac per year. Intel ship roughly ~220M PC CPU per year ( I think recent years saw the number trending back towards 250M) That is close to 10%. Not an insignificant number. Apple only use expensive Intel CPU. Lots of survey shows most $1000+ PC belongs to Apple. Most of the business Desktop, and Laptop uses comparatively cheap Intel CPU. i.e I would not be surprised the median price of Apple's Intel CPU purchase is at least 2x to total market median if not more. In terms of revenue that is 20% of Intel's consumer segment.

They used to charge a premium for being the leading edge Fab. You cant get silicon that is better than Intel. You are basically paying those premiums for having the best. Then Intel's Fab went from 2 years leading, to now 2 years behind, ( That is 4 years difference. ) all while charging the same price. And since Intel wants to keep its margin, it is not much of a surprise customers, ( Apple and Amazon ) looks for alternative.

Here is another piece on Amazon Graviton 2.

https://threader.app/thread/1274020102573158402

May be I should submit it to HN.

Apple owns the premium PC market, it’s Mac division is not only the most profitable PC company in the world, it might be more profitable than all the others combined.

It’s share of Intels most expensive desktop CPUs is much higher than its raw market share.

If macOS goes ARM and there's a sizable population of developers testing and fixing these issues constantly, the math changes in favor of Graviton and it would make it a no-brainer to pick the cheaper alternative once everything "just works".

I've been shifting my work to ARM based machine for some years now, mainly to reduce power consumption. One of my current projects - A problem validation platform[4](Go) has been running nicely on a ARM server(Cavium ThunderX SoCs) on scaleway; but weirdly scaleway decided to quit on ARM servers[5] sighting hardware issues which not many of the ARM users seem to have faced. Only ARM specific issue I faced with scaleway was that the reboot required power-off.

[1]cudf: https://gist.github.com/heavyinfo/da3de9b188d41570f4e988ceb5...

[2]Ray: https://gist.github.com/heavyinfo/aa0bf2feb02aedb3b38eef203b...

[3]Apache Arrow: https://gist.github.com/heavyinfo/04e1326bb9bed9cecb19c2d603...

[4]needgap: https://needgap.com

[5]Scaleway ditched ARM: https://news.ycombinator.com/item?id=22865925

I have a hunch our industry will need to increasingly deal with ARM environments.

What is the connection here ? ARM servers would be fine in a separate discussion. What does it have to do with Macs ? Macs aren't harbingers of anything. They have set literally no trend in the last couple of decades, other than thinness at all costs. If you mean that developers will use Gravitons to develop mac apps, why/how would that be ?

"Some people think that "the cloud" means that the instruction set doesn't matter. Develop at home, deploy in the cloud.

That's bull*t. If you develop on x86, then you're going to want to deploy on x86, because you'll be able to run what you test "at home" (and by "at home" I don't mean literally in your home, but in your work environment)."

So I would argue there is a strong connection.

I can see this making sense to Torvalds, being a low-level guy, but is it true for, say, Java web server code?

Amazon are betting big on Graviton in EC2. It's no longer just used in their 'A1' instance-types, it's also powering their M6g, C6g, and R6g instance-types.

https://aws.amazon.com/about-aws/whats-new/2019/12/announcin...

Another example is android apps development. Most developers use x86_64 CPU and run emulator using intel android image. While I don't have vast experience, I did write few apps and never had any issue because of arch mismatch.

High level languages mostly solved that issue.

Also note that there are some ARM laptops in the wild already. You can either use Windows or Linux. But I don't see that every cloud or android developer hunting for that laptop.

Going from ARM to X64 is even less likely to have issues as X64 is more permissive about things like unaligned access.

People are making far too big a deal out of porting between the two. Unless the code abuses undefined behaviors in ways that you can get away with on one architecture and not the other, there is usually no issue. Differences in areas like strong/weak memory ordering, etc., are hidden behind APIs like posix mutexes or std::atomic and don't generally have to be worried about.

The only hangup is usually vector intrinsic or ASM code, and that is not found in most software.

For higher level languages like Go or Java, interpreted languages like JavaScript or Python, or more modern languages with fewer undefined edge cases like Rust, there is almost never an issue.

This is just not a big deal unless you're a systems person (like Linus) or developing code that is really really close to the metal.

Thus I encourage everyone to target more than one platform as it makes the total better. This even though there are platform specific issues that won't happen on the other (like the compiler bug we found)

I would be surprised if Apple didn’t have internal prototypes of macos running on their own Arm chips for the last several years. Most of the macos / iOS code is shared between platforms, so it’s already well optimized for arm.

Usually making the application run in the host OS is much more productive than dealing with the emulator/simulator.

Unless you're talking about native code (and even then, I've written in the past about ways this can be managed more easily), then no, it really doesn't matter.

If you're developing in NodeJS, Ruby, .NET, Python, Java or virtually any other interpreted or JIT-ed language, you were never building for an architecture, you were building for a runtime, and the architecture is as irrelevant to you as it ever was.

Well I can't speak to some of the others... but Conda doesn't work at all on ARM today (maybe that will change with the new ARM Macs, though), which is annoying if you want to use it on, say, a Raspberry Pi for hobby projects.

Additionally, many scientific Python packages use either pre-compiled binaries or compile them at install-time, for performance. They're just Python bindings for some C or Fortran code. Depending on what you're doing, that may make it tricky to find a bug that only triggers in production.

Also one I've come across myself so I'm a bit disappointed I didn't call this out. So... kudos!

I think you are vastly underestimating how many people use Mac (or use Windows without using WSL) to develop for the cloud.

The things the Mac is good at:

1) It powers my 4k monitor very well at 60Hz

2) It switches between open lid and closed lid, and monitor unplugged / plugged in states consistently well.

3) It sleeps/wakes up well.

4) The built in camera and audio work well, which is useful for meetings, especially these days.

None of these things really require either x86 or Arm. So if a x86 based non-Mac laptop appeared that handled points 1-4 and could run Linux closer to our production environment I'd be all over it.

I started using my home Windows machine for development as a result of the lockdown and in all honesty I have fewer issues with it than I did with my work MacBook. Something is seriously wrong here.

Most of the teens to early 20 somethings I know are either buying or hoping to buy the latest Macs, iPads, iPhones and AirPods while most of the devs I know are on Linux or WSL but devs are a minority compared to the Average Joes who don't code but are willing to pay for nice hardware and join the ecosystem.

And as a byproduct perhaps they will work better for hip young consumers too. Or anyone else who is easily distracted by bright colours and simple pictures, which is nearly all of us.

The dominance of Macs for software development is a very US-centric thing. In Germany, there is no such Mac dominance in this domain.

And why not Java developers?

They seem pretty polar opposite in terms of culture, but all still turn up using a Macintosh.

Even the famous FailOverflow said in one of his videos he only bought a Mac since he saw that at conferences everyone had Macs so he thought that must mean they're the best machines.

Anecdotally, I've interviewed at over 12 companies in my life and only one of those issues Mac to its employees the rest were windows/Linux.

I've got anecdata that says different. My backend/cloud team has been pretty evenly split between Mac and Windows (with only one Linux on the desktop user). This is at a Java shop (with legacy Grails codebases to maintain but not do any new development on).

With Mac all your regular Mac software integrates well with the Unix world. XCode is not going to screw up my line endings. I don’t have to keep track of whether I am checking out a file from a Unix or Windows environment.

`git config --global core.autocrlf true`

That will configure git to checkout files with CRLF endings and change them to plain LF when you commit files.

They're not ancient, but are mostly ports of recent FreeBSD (or occasionally some other BSD) utilities. Some of these have a lineage dating back to AT&T/BSD Unix, but are still the (roughly) current versions of those tools found on those platforms, perhaps with some apple-specific tweaks.

You must be living in a different universe. What do you think the tens of thousands of developers at Google, Facebook, Amazon, etc etc etc are doing on their Macintoshes?

I can only speak of my experience at Google, but the Macs used by engineers here are glorified terminals, since the cloud based software is built using tools running on Google's internal Linux workstations and compute clusters. Downloading code directly to a laptop is a security violation (With an exception for those working on iOS, Mac, or Windows software)

If we need Linux on a laptop, there is either the laptop version of the internal Linux distro or Chromebooks with Crostini.

edit: https://insights.stackoverflow.com/survey/2020#technology-pr...

SSH to a linux machine ? I get that cloud software is a broad term that includes pretty much everything under the sun. My definition of cloud dev was a little lower level.

https://www.theregister.com/2020/05/24/linus_torvalds_adopts...

What point are you trying to make?

Yeah, I'm able to do remote debugging in a remote VM but the feedback loop is much longer, impacting productivity, morale and time to solve the bug, a lot of externalised costs that all engineers with reasonable experience are aware of. If I can develop my code on the same architecture that it'll be deployed my mind is much more in peace, when developing in x86_64 to deploy on ARM I'm never sure that some weird cross-architecture bug will pop up. No matter how good my CI/CD pipeline is, it won't ever account for real-world usage.

it's harder in all the way you describe, but it's much more likely the software will survive migrating to the next debian/centos release unchanged.

it all boils down to the temporal scale of the project.

I have to agree. It's not like we're all running Darwin on servers instead of Linux. Surely the kernel is a more fundamental difference than the CPU architecture.

Hahaha then you have not kept attention. Apple led the trend away from beige boxes. Style of keyboard used. Large track pads. USB. First to remove floppy drive. Both hardware, software and web design has been heavily inspired by Apple. Just look at icons used, first popularized by Apple.

Ubuntu desktop is strongly inspired by macOS. Operating system with drivers preloaded through update mechanism was pioneered by Apple. Windows finally seem to be doing this.

Being cheaper isn't enough here - Graviton needs to be faster and it needs to do that over generations. It needs to _sustain_ its position to become attractive. Intel can fix price in a heartbeat - they've done that in the past when they were behind. Intel's current fab issues does make this a great time to strike, but what about in 2 years? 3 years? 4? Intel's been behind before, but they don't stay there. Switching to Epyc Rome at the moment is an easy migration - same ISA, same memory model, vendor that isn't new to the game, etc... But Graviton needs a bigger jump, there's more investment there to port things over to ARM. Will that investment pay off over time is a much harder question to answer.

I agree to some extent but don’t underestimate how much this has changed in the cloud era: it’s never been cheaper to run multiple ISAs and a whole lot of stuff is running in environments where switching is easy (API-managed deployments, microservices, etc.) and the toolchains support ARM already thanks to phones/tablets and previous servers - so much code will just run on the JVM, high level languages like JavaScript Python, or low-level ones like Go and Rust with great cross-compiler support, etc. and hardware acceleration also takes away the need to pour engineer-hours into things like OpenSSL which might have blocked tons of applications.

At many businesses that is probably over the threshold where someone can say it’s worth switching the easy n% over since they’ll save money now and if the price/performance crown shifts back so can their workloads. AWS has apparently already done this for managed services like load-balancers and I’m sure they aren’t alone in watching the prices very closely. That’s what Intel is most afraid of: the fat margins aren’t coming back even if they ship a great next generation chip.

Some function gets vectorized and a variable shifts in or out of a register? Or one of your libraries gets rebuilt with LTO / LTCG and its functions are inlined.

If your code was wrong, any of that can ruin it and you're left trying to figure it out, and requiring your developers to use exactly one point release of Visual Studio and never, ever upgrade the SDK install.

I love working on ARM, but with the licensing model I'm not sure how much of a positive return Amazon would really be able to squeeze from their investment.

It also potentially brings them a truckload of future headaches if any of their cloud competitors raise anti-trust concerns down the road.

Beyond that, I think Apple, ARM and AMD get a lot of credit for their recent designs - a lot of which is due, but quite a bit of which should really go to TSMC.

The TSMC 7nm fabrication node is what's really driven microprocessors forward in the last few years, and hardly anyone outside of our industry has ever heard of them.

I don't know that Amazon couldn't transition to RISC-V if they needed to in a couple of years.

Microsoft controls a lot of lucrative business-centric markets. An advantage that seems to have helped MS Azure surpass AWS in marketshare. One of Microsoft's weaknesses is their in-ability to migrate away from x86 in any meaningful way. IBM could used Redhat to push the corporate server market away from x86 under the guise of lower operating costs, which could deal a tremendous blow to MS, leaving Amazon and Google with an opening to hit the Office and Azure market.

You overestimate the AWS customer base. Lots of them do silly things that cost them a lot of money.

It's often easy to test if scaling the instance size/count resolves a performance issue. If it does, you know you can fix the problem by burning money.

When you have reasonable certainty of the outcome spending money is easier than engineering resources.

And later it's easier for an engineer to justify performance optimizations, if the engineer can point to a lower cloud bill..

I'm not saying it's always a well considered balance, just that burning money temporarily is a valid strategy.

If only AWS had thousands of engineers to create a UX that makes cost info all upfront and easy to budget. Clearly it's beyond their capability /s

>I'm not saying it's always a well considered balance, just that burning money temporarily is a valid strategy.

Yes, but I bet a majority of AWS users don't want that to be the default strategy.

Why do you think that? Cloud means linux, apple does not even have a server OS anymore.

We are seeing some ARM devices entering the server space, but these have been in progress for ears and have absolutely nothing to do with apples (supposed) CPU switch.

One reason is to get the same environment they get in a cloud deployment scaled down. Another is Apple keeps messing with compatibility, most recent example being the code signing thing slowing down script executions.

Edit, this includes docker on Mac: it's not native.

> Guys, do you really not understand why x86 took over the server market?

> It wasn’t just all price. It was literally this “develop at home” issue. Thousands of small companies ended up having random small internal workloads where it was easy to just get a random whitebox PC and run some silly small thing on it yourself. Then as the workload expanded, it became a “real server”. And then once that thing expanded, suddenly it made a whole lot of sense to let somebody else manage the hardware and hosting, and the cloud took over.

Sure they have lower margins on desktop but these brings sh!t tons of cash, cash you will then use for research and to develop your next desktop and server CPUs. If I believe this page [1], consumer CPUs brought almost $10 billions in 2019... In comparison, server CPUs generated $7 billions of revenues... And these days, Intel has like >90% of that market.

Other players (Sun, HP, IBM, Digital, ...) were playing in a walled garden and couldn't really compete on any of that (except pricing because their stuff was crazy expensive).

So not only they were sharing the server market with Intel but Intel was most likely earning more than the sum of it all from their desktop CPUs... More money, more research, more progress shared between the consumer and server CPUs: rinse and repeat and eventually you will catch up. And also, they could sell their server CPUs for a lot less than their "boutique" competitors.

You just can't compete with a player which has almost unlimited funds and executes well...

[1] https://www.cnbc.com/2020/01/23/intel-intc-earnings-q4-2019....

People paid the IBM tax for decades. Intel is the new IBM. Wherever you look, Intel's lineup is not competitive and it is only surviving due to inertia.

[0] https://pharr.org/matt/blog/2018/04/30/ispc-all.html

[1] https://pharr.org/matt/blog/2018/04/28/ispc-talks-and-depart...

Yes, Spectrum and Meltdown definitely proves that...

I think I heard that argument already, in the 90's. But instead of Intel it was Sun, it was SGI, it was Digital, etc

The truth is that money wins and while there is an inertia, it's more like a Cartoon inertia where your guy won't fall from the cliff until he looks down. But then it's too late.

None of this is rocket science but it takes money and engineers time to make these things happen. Intel has more of both at the moment.

https://ark.intel.com/content/www/us/en/ark/products/205684/...

https://www.amd.com/en/products/cpu/amd-epyc-7h12

Epyc 7H12[1]:

- L1: two 32KiB L1 cache per core

- L2: 512KiB L2 cache per core

- L3: 16MiB L3 cache per core, but shared across all cores.

The L1/L2 cache aren't yet publicly available for any Cooper Lake processors, however the previous Cascade Lake architecture provided:

All Xeon Cascade Lakes[2]:

- L1: two 32 KiB L1 cache per core

- L2: 1 MiB L2 cache per core

- L3: 1.375 MiB L3 cache per core (shared across all cores)

Normally I'd expect the upcoming Cooper Lake to surpass AMD in L1, and lead further in L2 cache. However it looks like they're keeping the 1.375MiB L3 cache per core in Cooper Lake, so maybe L1/L2 are also unchanged.

0: https://www.hardwaretimes.com/cpu-cache-difference-between-l...

1: https://en.wikichip.org/wiki/amd/epyc/7h12

2: https://en.wikichip.org/wiki/intel/xeon_platinum/9282

Edit: Previously I showed EPYC having twice the L1 as Cascade Lake, this was a typo on my part, they're the same L1 per core.

There's been very few cloud benchmarks where 1P Epyc Rome hasn't beaten any offering from Intel, including 2P configurations. The L3 cache latency hasn't been a significant enough difference to make up the raw CPU count difference, and where L3 does matter the massive amount of it in Rome tends to still be more significant.

Which is kinda why Intel is just desperately pointing at a latency measurement slide instead of an application benchmark.

As another commenter pointed out, though, not all caches are equal. Unfortunately, I was not able to easily find access speeds for specific processors, so single-threaded benchmarks are the primary quantitative differentiator.

And I think Xeon L3 cache tops out at about 40MB, whereas Threadripper & Epyc go up to 256MB.

I don't think there's any Intel CPU's with more than 36MB L3?

IIRC the only people who buy those vs. the equivalent dual socket system are people with expensive software which is licensed per socket.

But Amdhal's Law shows us this doesn't make sense for most people.

Graviton is still almost an order of magnitude slower than a VPS of the same cost, which is around what the hardware and infra costs Amazon.

The limitation is not technical (as hackintoshes demonstrate); it’s legal.

That said, it would be great to be able to run Mac software (and iOS) on server-based VMs.

I just don’t think it will happen at scale, because lawyers.

what are my options if I want a ARM laptop with say good mobile processor performance close to the i7-8850H found in a 2018 mbp 15, 16GB RAM and 512GB NVME SSD to setup my day to day dev environment (Linux + Golang + C++ etc)?

surface x is the only ARM laptop you can easily purchase, but it is windows only and the processor is way too slow. there is also close to 0 response from app vendors to bring their apps to native ARM windows environment.

Apple fans just seem to be in denial about being late to the game.

I have to admit, the Apple's ARM processors will likely be significantly faster per core. But they are not the driver of the switch to ARM. If anything, Chromebooks were.

If you develop for Mac, chances are you want your CI to use the same target hardware, which means cloud ARM hardware to run Mac VMs.

Wishful thinking though, probably.

there's still a load of non scale-out services that the world depends upon.

https://amperecomputing.com/

Ever since Cavium gave up Arm server products and pivoted to HPC, there hasn't been a real Arm compititor.

Ampere is almost all ex-Intel.