(It explains why iOS does better with less ram than android, but the quote is specifically claiming this as a reason for 8GB ram to be acceptable)
Perhaps they never really needed to fit 32GB into their intel macs either. Some days after the glowing reviews; and strange comments about magic memory utilization; we now see comments concerned about SSD wear due to swap file usage.
If the applications and data structures are more compact in memory on the arm processors; it should be easy to test; you just need an intel mac; and an M1 mac running the same app on the same document and look at how much memory it uses.
So on customer PC ram is maybe more used as caching mechanisms or eaten away by poorly designed memory leak/garbage collection.
And if your GPU is able do to real time rendering on data heavy load maybe you need less caching of intermediate results as well.
2.My current production server is a PostgreSQL database on a 16GB RAM VM running on Debian (my boss is stingy). This doesn't prevent me from managing a 300GB+ data cluster with pretty decent performances and perform actual data analysis.
3.If Chrome sometimes use +8GB for a godsake webrowser the only explanation is poor design, there is no excuse.
It's possible that apps have been completely overhauled for a baseline M1 experience. Extremely, extraordinarily unlikely that anything remotely of the sort has happened, though. And since M1-equipped Macs don't have any faster IO than what they replaced (disk, network, and RAM speeds are all more or less the same), there wouldn't be any reason for apps to have done anything substantially difference.
Third, Marcel Weiher explains Apple’s obsession about keeping memory consumption under control from his time at Apple as well as the benefits of reference counting:
>where Apple might have been “focused” on performance for the last 15 years or so, they have been completely anal about memory consumption. When I was there, we were fixing 32 byte memory leaks. Leaks that happened once. So not an ongoing consumption of 32 bytes again and again, but a one-time leak of 32 bytes.
>The benefit of sticking to RC is much-reduced memory consumption. It turns out that for a tracing GC to achieve performance comparable with manual allocation, it needs several times the memory (different studies find different overheads, but at least 4x is a conservative lower bound). While I haven’t seen a study comparing RC, my personal experience is that the overhead is much lower, much more predictable, and can usually be driven down with little additional effort if needed.
ARC is not specific to M1, BUT have been widely used in ObjC & Swift for years AND is thus heavily optimized on M1 that perform "retain and release" way faster (even when emulating x86)
Perfect illustration of Apple software+hardware long term strategy.
>This quote doesn’t really cover why M1 macs are more efficient with less ram than intel macs? You’ve got a memory budget, it’s likely broadly the same on both platforms
But both Intel Macs and ARM Macs use RC. Both chips are running the same software.
Browsers are written in C++ and javascript has full-blown GC.
I don't see how refcounting gives you advantage over manual memory management for most users.
I am going to speculate now, but maybe, just maybe, if some of the silicon that apple has used on the M1 is used for compression/decompression they could be transparently compressing all ram in hardware. Since this offloaded from the CPUs and allows a compressed stream of data from memory, they achieve greater ram bandwidth, less latency and less usage for a given amount of memory. If this is the case I hope that the memory has ECC and/or the compression has parity checking....
Are you aware of any x86 chips that utilize this method?
Cheers
Making it cheaper to create and destroy objects with hardware acceleration, and to do many small, low-cost reclaims without eating all your CPU would be a magical improvement to the JVM, because you could constrain memory use without blowing out CPU. From what's described in TFA it sounds like the same is true for modern MacOS programming.
The JVM already makes it extremely cheap to create and destroy objects: creation is always ~free (just a pointer increment), and then destruction is copying, so very sensitive to memory bandwidth but done in parallel. If most of your objects are dying young then deallocation is "free" (amortized over the cost of the remaining live objects). Given the reported bandwidth claims for the M1 if they ever make a server version of this puppy I'd expect to see way higher GC throughput on it too (maybe such a thing can be seen even on the 16GB laptop version).
The problem with Java on the desktop is twofold:
1. Versions that are mostly used don't give memory back to the OS even if it's been freed by the collector. That doesn't start happening by default until like Java 14 or 15 or so, I think. So your memory usage always looks horribly inflated.
2. If you start swapping it's death because the GC needs to crawl all over the heap.
There are comments here saying the M1 systems rely more heavily on swap than a conventional system would. In that case ARC is probably going to help. At least unless you use a modern pauseless GC where relocation is also done in parallel. Then pausing background threads whilst they swap things in doesn't really matter, as long as the app's current working set isn't swapped out to compensate.
But when swap hits 8-9 Gb, it’s effects start to get very noticeable.
Besides, a lot of memory usage is in web browsers, which must use garbage collection.
Looking at the reviews of M1 Macs, those systems are still responsive and making forward progress at a “memory pressure” that would make my x86 Mac struggle in a swap storm. It seems to come down to very fast access to RAM and storage, large on-die caches, and perhaps faster memory compression.
2 more points:
- All the evidence I've seen is gifs of people opening applications in the dock, which is... not impressive. I can do that already, apps barely allocate at all when they open to "log in to iCloud" or "Safari new tab". And don't we see that literally every time Apple launches Mac hardware? Sorry all tech reviewers everywhere, try measuring something.
- I think the actual wins come from the zillion things Apple has done in software. Like: memory compression, which come to think of it might be possible to do in hardware. Supposedly a lot of other work/tuning done on the dynamic pager, which is maybe enabled by higher bandwidth more than anything else.
Fun fact: you can stress test your pager and swap with `sudo memory_pressure`. Try `-l critical`. I'd like to see a benchmark comparing THAT under similar conditions with the previous generation.
I'm curious about FP/vector performance, but I'm pretty sure it's fine. I'm definitely eyeing a MBP myself! 20 hours of video playback? Crazy...
Because the M1 is similar to the chips used in iOS, hence the comparison is not inappropriate.
>The memory bandwidth on the new Macs is impressive. Benchmarks peg it at around 60GB/sec–about 3x faster than a 16” MBP. Since the M1 CPU only has 16GB of RAM, it can replace the entire contents of RAM 4 times every second. Think about that…
Or that's how I understand this, I don't actually own M1 Mac.
https://www.anandtech.com/show/16252/mac-mini-apple-m1-teste...
> Besides the additional cores on the part of the CPUs and GPU, one main performance factor of the M1 that differs from the A14 is the fact that’s it’s running on a 128-bit memory bus rather than the mobile 64-bit bus. Across 8x 16-bit memory channels and at LPDDR4X-4266-class memory, this means the M1 hits a peak of 68.25GB/s memory bandwidth.
The point of the memory bandwidth is so that it never has to swap to disk in the first place.
What? How does memory bandwidth obviate the need for disk swapping?
It is. We know they're using a unified memory architecture, they pointed it out in the presentation.
The reason GDDR isn't typically used for system RAM is it's higher latency & more power hungry. Like, the GDDR6 memory on a typical discreet card uses more power than the an entire M1-powered Mac Mini power hungry.
What "normal" laptop has that?
The latency appears real to me.
Separately from that x86's TSO-ish memory model also imposes a performance cost whether your algorithm needs those guarantees or not. Code sometimes relies on those guarantees without knowing it. Absent hardware support you would need to insert ARM atomics in translated code to preserve those guarantees which on most ARM CPUs would impose a lot of overhead. The M1 allows Rosetta to put the CPU into a memory ordering mode that preserves the expected memory model very efficiently (as well as using 4K page size for translated processes).
They are fast for atomics but still far, far slower than the equivalent non-atomic operation. An add operation takes around half a cycle (upper bound here - with how wide the firestorm core is an add operation is almost certainly less than half a cycle). At 1ghz a cycle is 1 nanosecond. The M1 runs at around 3ghz. So you're still talking the atomic operation being >10x slower than non-atomics.
Which should not be surprising at all. Apple didn't somehow invent literal magic here. They still need coherency across 8 cores, which means at a minimum L1 is bypassed for the atomic operation. The L2 latency is very impressive, contributing substantially to that atomic operation performance. But it's still coming at a very significant cost. It's very, very far from free. There's also no ARM vs. x86 difference here, since the atomic necessarily forces a specific memory ordering guarantee that's stricter than x86's default. Both ISAs are forced to do the same thing and pay the same costs.
How did you arrive at this number?
It's in the post. Half a cycle for an add or less, and cycles are every 1/3 nanosecond. So upper bound for an add would be around 1/6th a nanosecond. Likely less than that still yet, since the M1 is probably closer to an add in 1/8th a cycle not 1/2. Skylake by comparison is at around 1/4th a cycle for an add, and since M1's IPC is higher it's not going to be worse at basic ALU ops.
6 nanoseconds @ 3ghz is 18 cycles. That's on the slow end of the spectrum for a CPU instruction.
I'm not an EE expert and I haven't torn apart an M1, but Occams's Razor would suggest it's unlikely they made specialized hardware for NSObjects specifically. Other ARC systems on the same hardware would likely see similar benefits.
Kotlin/Native lets us do this comparison somewhat directly. The current and initial versions used reference counting for memory management. K/N binaries were far, far slower than the equivalent Kotlin programs running on the JVM and the developer had to deal with the hassle of RC (e.g. manually breaking cycles). They're now switching to GC.
The notion that GC is less memory efficient than RC is also a canard. In both schemes your objects have a mark word of overhead. What does happen though, is GC lets you delay the work to deallocate from memory until you really need it. A lot of people find this quite confusing. They run an app on a machine with plenty of free RAM, and observe that it uses way more memory than it "should" be using. So they assume the language or runtime is really inefficient, when in reality what's happened is that the runtime either didn't collect at all, or it collected but didn't bother giving the RAM back to the OS on the assumption it's going to need it again soon and hey, the OS doesn't seem to be under memory pressure.
These days on the JVM you can fix that by using the latest versions. The runtime will collect and release when the app is idle.
Specifically, as I understood it is that Apple software (written in objective C/Swift) uses a lot of retain/release (or Atomic Reference Counting) on top of manual memory, for memory management rather than other forms of garbage collection (such as those found in Java/C#), which gives Objective C programs a lower memory overhead (supposedly). This is why the iPhone ecosystem is able to run so much more snappier than the Android ecosystem.
That said, I don't see how that translates to lower memory usage than x86 programs. I think the supporting quotes he used for that point are completely orthogonal. I don't have an M1 mac, but I believe the same program running on both machines should use the same amount of memory.
I think you can reach a lot more than that. Presumably, on Intel they use something like LZO or LZ4, since it compresses/decompresses without too much CPU overhead. But if you have dedicated hardware for something like e.g. Brotli or zstd, one could reach much higher compression ratios.
Of course, this is assuming that memory can be compressed well, but I think this is true in many cases. E.g. when selecting one of the program/library files in the squash benchmarks:
https://quixdb.github.io/squash-benchmark/
you can observe higher compression ratios for e.g. Brotli/gzip/deflate than LZO/LZ4.
Edit: apparently, this isn't common knowledge.
GC vs RC is not a trivial comparison to make, but overall there are good reasons new systems hardly use RC (Objective-C dating back to the 90s isn't new). Where RC can help is where you have a massive performance cliff on page access, i.e. if you're swapped to disk. Then GC is terrible because it'll try and page huge sections of the heap at once where as RC is way more minimal in what it touches.
But in most other scenarios GC will win a straight up fight with an RC based system, especially when multi-threading gets involved. RC programs just spend huge amounts of time atomically incrementing and decrementing things, and rummaging through the heap structures, whereas the GC app is flying along in the L1 cache and allocations are just incrementing a pointer in a register. The work of cleaning up is meanwhile punted to those spare cores you probably aren't using anyway (on desktop/mobile). It's tough to beat that by hand with RC, again, unless you start hitting swap.
If M1 is faster at memory ops than x86 it's because they massively increased memory bandwidth. In fact I'd go as far as saying the CPU design is probably not responsible for most of the performance increase users are seeing. Memory bandwidth is the bottleneck for a lot of desktop tasks. If M1 core is say 10% faster than x86 but you have more of them and memory bandwidth is really 3x-4x larger, and the core can keep far more memory ops in flight simultaneously, that'll explain the difference all by itself.
On the other hand, that same paper shows that for every single one of their tested workloads, the generational GC outperforms manual memory management. Now obviously, you could do better with manual memory management if you took the time to understand the memory usage of your application to reduce fragmentation and to free entire arenas at a time, but for applications that don't have the developer resources to apply to that (the vast majority), the GC will win.
I'm not saying that better memory management is the reason Android wins these launch to interactivity benchmarks because the difference is so stark relative to the hardware performance that memory management isn't nearly enough to explain it, but it does contribute to it. (My own guess is that most of the performance difference comes from smarter process initialization from usage data. Apple is notoriously bad at using data for optimization.)
Apple has decades of proven experience producing and shipping massively over engineered systems. I believe em when they say these processors do ARC natively.
Hopefully I can clear up the discussion a little:
Q: Does reference counting 'use' less RAM than GC?
A: Yes (caveats etc. go here, but your question is a good explanation)
Q: Does the M1 in and of itself require less RAM than x86 processors?
A: No
Q: So why are people talking about the M1 and its RAM usage as if it's better than with x86?
A: It's really just around the faster reference counting. MacOS was already pretty efficient with RAM.
I'd like to propose tokamak-teapot's formula for hardware purchase:
Minimum RAM believed to be required = actual amount of RAM required * 2
N.B. I am aware that a sum that's greater than 16GB doesn't magically become less than 16GB, but it is somewhat surprising how well MacOS performs when it feels like RAM should be tight, so I'd suggest borrowing a Mac or making a Hackintosh to experience this if you're anxious about hitting the ceiling.
Then they pivoted into automating retain/release patterns from Cocoa and sold it, Apple style, as a victory of RC over tracing GC, while moving the GC related docs and C related workarounds into the documentation archive.
Operative word: tried. GC was an optional desktop-only component deprecated in Mountain Lion, which IIRC has not been accepted on MAS since 2015 was removed entirely from Sierra.
Without going into irrelevant weeds, "apple has always used refcounting everywhere" is a much closer approximation.
Which then in Apple style ("you are holding it wrong") turned it around in a huge marketing message, while hiding away the tracing GC efforts.
That's not exactly relevant to the subject at hand of what memory-management method software usually uses on macos.
> Which then in Apple style ("you are holding it wrong") turned it around in a huge marketing message, while hiding away the tracing GC efforts.
Hardly?
And people are looking to refcounting as a reason why apple software is relatively light on memory, which is completely fair and true and e.g. generally assumed as one of the reasons why ios devices fare well with significantly less ram than equivalent android devices. GCs have significant advantages, but memory overhead is absolutely one of the drawbacks.
https://github.com/ixy-languages/ixy-languages
And the fact that M1 has special instructions dedicated to optimize RC,
https://blog.metaobject.com/2020/11/m1-memory-and-performanc...
Memory overhead in languages with tracing GC (RC is a GC algorithm) only happens in languages like Java without support for value types.
If the language supports value types, e.g. D, and there is still memory overhead versus RC, then fire the developers or they better learn to use the language features available on their plate.
> https://github.com/ixy-languages/ixy-languages
This shows latency, not memory consumption, as far as I can tell.
> If the language supports value types, e.g. D, and there is still memory overhead versus RC, then fire the developers or they better learn to use the language features available on their plate.
Memory overhead of certain types of garbage collectors (notably generational ones) is well-known and it's specified relative to the size of the heap that they manage. Using value types is of course a valid point, regarding how you should use the language, but it doesn't change the overhead of the GC, it just keeps the heap it manages smaller. If the overhead was counted against the total memory use of a program, then we wouldn't be talking about the overhead of the garbage collector, but more about how much the garbage collector is actually used. Note that I'm not arguing against tracing GCs, only trying to keep it factual.
But I think in general you could say that Apple has focused more on optimizing their OS for memory usage than the competition may have done. Android uses Java which eats memory like crazy and I suspect C# is not that much better being a managed and garbage collected language. Not sure how much .NET stuff is used on Windows, but I suspect a lot.
macOS in contrast is really dominated by Objective-C and Swift which does not use these memory hungry garbage collection schemes, nor require JIT compilation which also eats memory.
C# is better than JVM in that it has custom value types.
Say you want to allocate an array of points in Java you basically have to allocate array[pointer] all pointing to tiny 8 byte objects (for eg. 32 bit float x and y coords) + the overhead of object header. If you use C# and structs it just allocates a flat array of floats with zero overhead.
Not only do you pointlessly use memory, you have indirection lookup costs, potential cache misses, more objects for GC to traverse, etc. etc.
JVM really sucks at this kind of stuff and so much of GUI programming is passing around small structs like that for rendering.
FWIW I think they are working on some proposal to add value types to JVM but that probably won't reach Android ever.
I can attest that structs use less memory however IIRC they don't have methods so no GetHashCode() which made them way too slow to insert in a HashSet or Dictionary.
In the end I used regular objects in a Dictionary. RAM usage was a bit higher than structs (not unbearably so) but speed improvement was massive.
You can and should implement IEquatable on a struct, especially if you plan on placing them in a hashset - the default implementation will use reflection and will be slow but it's easy to override.
Given that the M1 chip was designed to better support reference counting, it makes sense that doing the same for HC could lead to a benefit
Hi folks!
It looks like my blog post[1] was the primary source for this (it's referenced both by this post and by the Gruber post), and to be clear, I did not claim that this helps ARM Macs use less RAM than Intel Macs. I think John misunderstood that part and now it has blown up a bit...
I did claim that this helps Macs and iPhones use less RAM than most non-Apple systems, as part of Apple's general obsessiveness about memory consumption (really, really obsessive!). This part of the puzzle is how to get greater convenience for heap allocation.
Most of the industry has settled on tracing GCs, and they do really well in microbenchmarks. However, they need a lot of extra RAM to be competitive on a system level (see references in the blog post). OTOH, RC trends to be more frugal and predictable, but its Achilles heel, in addition to cyclic references, has always been the high cost of, well, managing all those counts all the time, particularly in a multithreaded environment where you have to do this atomically. Turns out, Apple has made uncontented atomic access about as fast as a non-atomic memory access on M1.
This doesn't use less RAM, it decreases the performance cost of using the more frugal RC. As far as I can tell, the "magic" of the whole package comes down to a lot of these little interconnecting pieces, your classic engineering tradeoffs, which have non-obvious consequences over there and then let you do this other thing over here, that compensates for the problem you caused in this other place, but got a lot out etc. Overall, I'd say a focus on memory and power.
So they didn't add special hardware for NSObject, but they did add special hardware that also tremendously helps NSObjet reference counting. And apparently they also added a special branch predictor for objc_msgSend(). 8-). Hey, 16 billion transistors, what's a branch predictor or two among friends.. ¯\_(ツ)_/¯
[1] https://blog.metaobject.com/2020/11/m1-memory-and-performanc...
RAM capacity is just RAM capacity. Possibly Swift-made apps uses less RAM compared to other apps, but microarchitecture shouldn't be matter.
My guess it's mostly faster swapping.
Microarchitecture could help, perhaps by making context switches faster.
But it could also be custom peripheral/DMA logic for handling swapping between RAM and NVM.
I think it makes sense.. NVM should be fast enough that RAM only needs to act as more of a cache. But existing architectures have a lot of legacy of treating NVM like just a hard drive. Intel is also working on this with its Optane related architecture work.
You could also do on-the-fly compression of some kinds of data to/from RAM. But I havent heard any clues that M1 is doing that, and you'd need applications to give hints about what data is compressible.
Most experiment with 8GB M1 Macs I've seen so far (on YouTube) seems to start slowing down once the data cannot fit in a RAM, although the rest of the system remain responsive e.g. 8K RED RAW editing test. In the same test with 4K RED RAW there were some shuttering on the first playback but subsequent playback were smooth, which I guessed it was a result of swap being moved back into a RAM.
My guess would be they've done a lot of optimization on swap, making swapping less of an performance penalty (as ridiculous as it sounds, I guess they could even use Neural Engine to determine what should be put back into RAM at any given moment to maximize responsiveness.)
macOS has been doing memory compression since Mavericks using WKdm algorithm, but they also support Hybrid Mode[1] on ARM/ARM64 using both WKdm and a variant of LZ4 for quite some time (WKdm compress much faster than LZ4). I wouldn't be too surprised if M1 has some optimization for LZ4. So far I haven't seen anybody tested it.
It might be interesting to test M1 Macs with vm_compressor=2 (compression with no swap) or vm_compressor=8 (no compression, no swap)[2] and see how it runs. I'm not sure if there's a way to change bootargs on M1 Macs, though.
[1]: https://github.com/apple/darwin-xnu/blob/a449c6a3b8014d9406c...
[2]: https://github.com/apple/darwin-xnu/blob/a449c6a3b8014d9406c...
combined it should make a big difference to swapping
But the much faster SSD to RAM transfer for the M1 means that shuffling stuff in and out of RAM is much faster meaning RAM matters less.
The transfer speeds of the M1 SSDs have been benched at 2.7GB/s - about the same speed as mid-range NVMe SSDs (my ADATA SX8200 Pro and Sabrent Rocket are both faster and go for about $120/TB).
I expected SSD is way fast, but benchmark says its SSD is about below 3000MB/s RW, not very fast but usual Gen3 x4 speed.
That said I’d still be very hesitant buying a 8 GB M1 Mac. When my iMac only had 8 GB it was a real pain to use. Increasing my iMac’s memory to 24 GB made it usable.
Intellij has a ton of features but it's pretty heavyweight because of it; I'd like a properly built native IDE with user experience speed at the forefront. That's what I loved about Sublime Text. But I also like an IDE for all the help it gives me, things that the alternative editors don't do yet.
I've used VS Code for a while as well, it's faster than Atom at least but it's still web technology which feels like a TON of overhead that no amount of hardware can compensate for.
I've heard of Nova, but apparently I installed it and forgot to actually work with the trial so I have no clue how well it works. I also doubt it works for my particular needs, which intellij doesn't have a problem with (old codebase with >10K LOC files of php 5.2 code and tons of bad JS, new codebase with Go and Typescript / React).
The native Swift retain (swift_retain above) seems to be somewhere inside this mess: https://github.com/apple/swift/blob/main/stdlib/public/runti...
The short answer for why it can’t just be an increment is because the reference count is stored in a subset of the bits of the isa pointer, and when the reference count grows too large it has to overflow into a separate sidetable. So it does separate load and CAS operations in order to implement this overflow behavior.
But they learned the lesson from SOAR (Smalltalk On A RISC) and did not follow the example of LISP machines, Smalltalk machines, Java machines, Rekursiv, etc. and build specialized hardware and instructions. The benefits of the specialization are much, much less than expected, and the costs of being custom very high.
Instead, they gave the machine large caches and tweaked a few general purpose features so they work particularly well for their preferred workloads.
I wonder if they made trap on overflow after arithmetic fast.
This is two operations and in-between the two -- if and only if the respective memory location is shared between multiple cores or caches -- some form of synchronization must occur (like locking a bank account so you can't double draft on two ATMs simultaneously).
Now the way this is implemented varies a bit.
Apple controls most of the hardware ecosystem, programming languages, binary interface, and so on meaning there is opportunity for them to either implement or supplement ARM synchronization or atomicity primitives with their own optimizations.
There is nothing really preventing Intel from improving here as well -- it is just a easier on ARM because the ISA has different assumptions baked in, and Apple controls everything up the stream, such as the compiler implementations.
1. "weak" memory ordering (atomic Aquire/Release loads/stores)
2. low memory latencies between cache and system memory (so dirty pages in caches are faster updated etc.)
3. potential a coherency impl. optimized for this kind of atomic access (purely speculative: e.g. maybe sometimes updating less then a page in a cache when detecting certain atomic operations which changed a page or maybe wrt. the window marked for exclusive access in context of ll/sc-operations and similar)
Given that it's common for a object to stay in the same thread I'm not sure how much 2. matters for this point (but it does matters for general perf.). But I guess there is a lot in 3. where especially with low latency ram you might be able to improve performance for this cases.
I roughly understand how refcounting causes extra damage to cache coherency: anywhere that a refcount increment is required, you mutate the count on an object before you use it, and then decrement it later. Often times, those counting operations are temporally distant from the time that you access the object contents.
I do not really understand the "pipeline bubbles" part, and am curious if someone can elaborate.
Reading on in the wiki page, they talk about weak references (completely different than weak memory ordering referenced above). This reminds me that Cocoa has been making ever more liberal use of weak references over the years, and a lot of iOS code I see overuses them, particularly in blocks. I last looked at the objc implementation years ago, but it was some thread safe LLVM hash map split 8 or 16 ways to reduce lock contention. My takeaway was roughly, "wow that looks expensive". So while weak refs are supposed to be used judiciously, and might only represent 1% or less of all refs, they might each cost over 100x, and then I could imagine all of your points could be significant contributors.
In other words, weak references widen the scope of this guessing game from just "what chip changes improve refcounting" to "what chip changes improve parallelized, thread safe hash maps."
For the last few decades the industry has generally believed that GC lets code run faster, although it has drawbacks in terms of being wasteful with memory and unsuitable for hard-realtime code. Refcounting has been thought inferior, although it hasn't stopped the Python folks and others from being successful with it. It sounds like Apple uses refcounting as well and has found a way to improve refcounting speed, which usually means some sort of specific silicon improvement.
I'd speculate that moving system memory on-chip wasn't just for fewer chips, but also for decreasing memory latency. Decreasing memory latency by having a cpu cache is good, but making all of ram have less latency is arguably better. They may have solved refcounting hot spots by lowering latency for all of ram.
From Apple's site:
"M1 also features our unified memory architecture, or UMA. M1 unifies its high-bandwidth, low-latency memory into a single pool within a custom package. As a result, all of the technologies in the SoC can access the same data without copying it between multiple pools of memory." That is paired with a diagram that shows the cache hanging off the fabric, not the CPU.
That says to me that, similar to how traditionally the cpu and graphics card could access main memory, now they have turned the cache from a cpu-only resource into a shared resource just like main memory. I wonder if the GPU can now update refcounts directly in the cache? Is that a thing that would be useful?
The M1 Mac's seem to be 8 channels x 16 bits, which is the same bandwidth as a desktop (although running the ram at 4266 MHz is much higher than usual). The big win is you can have 8 cache misses in flight instead of 2. With 8 cores, 16 GPU cores, and 16 ML cores I suspect the M1 has more in flight cache misses than most.
The DDR4 bus is 64-bit, how can you have a 128-bit channel??
Single channel DDR4 is still 64-bit, it's only using half of the bandwidth the CPU supports. This is why everyone is perpetually angry at laptop makers that leave an unfilled SODIMM slot or (much worse) use soldered RAM in single-channel.
> The big win is you can have 8 cache misses in flight instead of 2
Only if your cache line is that small (16 bit) I think? Which might have downsides of its own.
Less familiar with the normal on laptops, but most desktop chips from AMD and Intel have two 64 bit channels.
> Which might have downsides of its own.
Typically for each channel you send an address, (a row and column actually), wait for the dram latency, and then get a burst of transfers (one per bus cycle) of the result. So for a 16 bit wide channel @ 3.2 Ghz with a 128 byte cache line you get 64 transfers, one ever 0.3125 ns for a total of 20ns.
Each channel operates independently, so multiple channels can each have a cache miss in flight. Otherwise nobody would bother with independent channels and just stripe them all together.
Here's a graph of cache line throughput vs number of threads.
https://github.com/spikebike/pstream/blob/master/png/apple-m...
So with 1,2 you see an increase in throughput, the multiple channels are helping. 4 threads is the same as two, maybe the L2 cache has a bottleneck. But 8 threads is clearly better than 4.
Yeah, I'm saying you can't magically unify them into a single 128-bit one. If you only use a single channel, the other one is unused.
I've seen similar on Intel servers, but not recently. This isn't however typically something you can do at runtime, just boottime, at least as far as I've seen.
But doesn't that only help if you have parallel threads doing independent 16 bit requests? If you're accessing a 64 bit value, wouldn't it still need to occupy four channels?
So striping a caching line across multiple channels goes increase bandwidth, but not by much. If the dram latency is 70ns (not uncommon) and your memory is running at 3.2 GHz on a single 64 bit wide channel you get 128 bytes in 16 transfers. 16 transfers at 3.2GHz = 5ns. So you get a cache line back in 75ns. With 2 64 bit channels you can get 2 cache lines per 75ns.
So now with a 128 bit wide channel (twice the bandwidth) you wait 70ns then get 8 transfers @ 3.2GHz = 2.5ns. So you get a cache line back in 72.5ns. Clearly not a big difference.
So the question becomes for a complicated OS with a ton of cores do you want one cacheline per 72.5ns (the stripped config) or two cachlines per 75ns (the non-stripped config).
In the 16 bit 8 channel (assuming the same bus speed and latency) you get 8 cacheline per 90ns. However not sure what magic apple has but I'm seeing very low memory latencies on the M1, on the order of 33ns! With all cores busy I'm seeing cacheline througput of a cacheline per 11ns or so.
sounds realy cool
I'm not familiar with MacOs, are the apps there mostly managed code? Even if they were and even if refcounting on Mac is that much faster than refounting on PC - refcounted code would still lose to manual memory management on average.
So low latency of the cache to system RAM can help here, at least for cases where the Rc is shared between threads. But also if the thread is not shared between threads but the thread is moved to a different CPU. Still it's probably not the main reason.
Given how atomic (might) be implemented on ARM and that the cach and memory is on the same chip my main guess is that they did some optimizations in the coherency protocol/implementation (which keeps the memory between caches and the system memory/RAM coherent). I believe there is a bit of potential to optimize for RC, i.e. to make that usage pattern of atomics fast. Lastly they probably take special care that the atomic related instructions used by Rc are implemented as efficient as possible (mostly fetch_add/fetch_sub).
> which keeps the memory between caches and the system memory/RAM coherent Isn't this already true of every multi-core chip ever designed; the whole point of coherency is to keep the RAM/memory coherent between all the cores and their caches.
> Isn't this already true of every multi-core chip ever designed;
Yes, I just added the explanation of what coherency is in this context as I'm not sure how common the knowledge about it is.
The thing is there are many ways how you can implement this (and related things) with a number of parameters involved which probably can be tuned to optimize for typical RC's usage of atomic operations. (Edit: Just to be clear there are constraints on the implementation imposed by it being ARM compatible.)
A related example (Not directly atomic fetch add/sub and not directly coherency either) would be the way LL/SC operations are implemented. Mainly on ARM you have a parameter of how large the memory region "marked for exclusive access" (by an LL-load operation) is. This can have mayor performance implications as it directly affects how likely a conditional store fails because of accidental inference.
I'm curious to understand this. Is this because of a specific instruction set support, or just the overall unified memory architecture?
For the price, it better run circles and squares. It should cook my dinner too.
https://www.androidcentral.com/cheapest-iphone-has-more-powe...
https://www.androidauthority.com/iphone-se-vs-most-powerful-...
Look, want to know how M1 achieve its result? Easy. Apple is first with a 5nm chips. Look in the past: every CPU maker gains both speed and power efficiency when going down a manufacturing node.
Intel CPU were still using a 14nm node (although they called 12+++) while Apple M1 is now at 5nm. According to this [1] chart, that's a transistor density at least 4x.
Not saying Apple has no CPU design chops, They've been at it for their phones for quite a while. But people are just ignoring the elephant in the room: Apple gives TSMC a pile of cash to be exclusive for mass production on their latest 5nm tech.
[1] https://www.techcenturion.com/7nm-10nm-14nm-fabrication#nbspnbspnbspnbsp7nm_vs_10nm_vs_12nm_vs_14nm_Transistor_DensitiesA reference counting strategy would be more efficient in processor utilization compared to garbage collection as it does not need to perform processor intensive sweeps through memory identifying unreferenced objects. So reference counting trades memory for processor cycles.
It is not true that garbage collection requires more ram to achieve equivalent performance. It is in fact the opposite. For programs with identical object allocations, a GC based system would require less memory, but would burn more CPU cycles.
I think it’s the reverse.
Firstly, garbage collection (GC) doesn’t identify unreferenced objects, it identifies referenced objects (GC doesn’t collect garbage). That’s not just phrasing things differently, as it means that the amount of garbage isn’t a big factor in the time spent in garbage collection. That’s what makes GC (relatively) competitive, execution-time wise. However, it isn’t competitive in memory usage. There, consensus is that you need more memory for the same performance (https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf: with five times as much memory, an Appel-style generational collector with a non-copying mature space matches the performance of reachability-based explicit memory management. With only three times as much memory, the collector runs on average 17% slower than explicit memory management)
(That also explains why iPhones can do with so much less memory than phones running Android)
Secondly, the textbook implementation of reference counting (RC) in a multi-processor system is inefficient because modifying reference counts requires expensive atomic instructions.
Swift programs spend about 40% of their time modifying reference counts (http://iacoma.cs.uiuc.edu/iacoma-papers/pact18.pdf)
So, reference counting gets better memory usage at the price of more atomic operations = less speed.
That last PDF describes a technique that doubles the speed of RC operations, decreasing that overhead to about 20-25%.
It wouldn’t surprise me if these new ARM macs use a similar technique to speed up RC operations.
It might also help that the memory model of ARM is weaker than that of x64, but I’m not sure that’s much of an advantage for keeping reference counts in sync across cores.
True, reference counting stores references… but garbage collection stores garbage, which is typically bigger than references :)
(Unless you’re thinking of a language where the GC gets run after every instruction - but I’m not aware of any that do that, all the ones I know of run periodically which gives garbage time to build up)
I've seen this with java where the memory usage graph looks like a sawtooth, with 100s of MB being allocated and then freed up a couple of seconds later.
In the tracing GCs I have seen, an "object header" must be stored with every object in the system; the GC needs it to know which parts of the object are references which should be traced. So while reference counting needs extra space to store the reference count, tracing GC needs extra space to store the object header.
> Retain and release are tiny actions that almost all software, on all Apple platforms, does all the time. ….. The Apple Silicon system architecture is designed to make these operations as fast as possible. It’s not so much that Intel’s x86 architecture is a bad fit for Apple’s software frameworks, as that Apple Silicon is designed to be a bespoke fit for it …. retaining and releasing NSObjects is so common on MacOS (and iOS), that making it 5 times faster on Apple Silicon than on Intel has profound implications on everything from performance to battery life.
> Broadly speaking, this is a significant reason why M1 Macs are more efficient with less RAM than Intel Macs. This, in a nutshell, helps explain why iPhones run rings around even flagship Android phones, even though iPhones have significantly less RAM. iOS software uses reference counting for memory management, running on silicon optimized to make reference counting as efficient as possible; Android software uses garbage collection for memory management, a technique that requires more RAM to achieve equivalent performance.