Re:Base64 encoding and decoding at almost the speed of a mem...
by ArtWomb on Wednesday November 06, 2019 @09:07AM
(#21462691)
Looks like model number 06/66 or better in cpuinfo. Both AWS and Gcloud standard vm instances appear to be Skylake ;(
Honestly, base64 has always been pretty fast for me, even large bitmaps >100kb. Compared to net latency it may be imperceptible to end user. But it still makes a terrific reference paper!
Re:Base64 encoding and decoding at almost the speed of a mem...
by jng on Wednesday November 06, 2019 @03:49AM
(#21460753)
Plus, if my knowledge is properly up-to-date, no AMD processor supports those instructions, not even any member the recent and famously super efficient Ryzen family processors.
Re:Base64 encoding and decoding at almost the speed of a mem...
by brazzy on Wednesday November 06, 2019 @02:59AM
(#21460563)
Because those are commodity processors nonetheless. The distinction relevant for a research paper is between mainstream processors you can just buy vs. custom made ones, not about market penetration.
Re:Base64 encoding and decoding at almost the speed of a mem...
by dickeytk on Wednesday November 06, 2019 @10:22AM
(#21463405)
I still think "commodity" is not the correct word even in that context. Mass-market, or commercially available might be better. How can it be a commodity when there is only one manufacturer?
Re:Base64 encoding and decoding at almost the speed of a mem...
by penagwin on Wednesday November 06, 2019 @11:18AM
(#21464003)
Just going off wikipedia (also other sources support the definition):
> In economics, a commodity is an economic good or service that has full or substantial fungibility: that is, the market treats instances of the good as equivalent or nearly so with no regard to who produced them.
Basically it's a common good, that is sold and purchased, with no other modifications required for this use-case. This commodity is currently low in supply, but is still a commodity.
Re:Base64 encoding and decoding at almost the speed of a mem...
by bloomer on Wednesday November 06, 2019 @11:38AM
(#21464205)
It’s missing the fungibility where it is produced by multiple suppliers and can be treated equivalently by the end buyer. Large grade A eggs are a commodity as it doesn’t matter which farm produced them. A specific set of processors from Intel are not a commodity since there are no fungible equivalents. Widely available would probably be a better description in this case.
Re:Base64 encoding and decoding at almost the speed of a mem...
by dickeytk on Thursday November 07, 2019 @12:44AM
(#21470758)
I would argue processors are never commodities anymore (maybe back in the 486 days it was closer). You can't swap an intel for an amd processor into the same motherboard ever today. They don't have the same performance characteristics from one to another. You could have 2 "3.5Ghz 4-core processors" that have wildly different real-world performance. IMO this is the opposite of "fungible"
I think hard drives, memory, power supplies are much closer to being fungible.
Re:Base64 encoding and decoding at almost the speed of a mem...
by londons_explore on Wednesday November 06, 2019 @05:07AM
(#21461140)
If base64 decoding speed makes a difference in your application, you should be considering why you are transmitting data in base64, a neat hack from the 1970's, which is non-human-readable, wastes 30% of the network bandwidth, kills compression, wastes RAM, is typically hard to random-access, and is generally a bad idea all round.
Re:Base64 encoding and decoding at almost the speed of a mem...
by dathinab on Wednesday November 06, 2019 @05:28AM
(#21461237)
I agree, IMHO is the biggest design fail and annoyance of JSON that it has no good way to contain binary data.
The problem start with knowing the encoding (base64, for small sizes sometimes hex, for one or two bytes sometimes arrays with all there ambiguities).
Then that you can't easily differentiate between a string a bytearray anymore (through you often shouldn't need to).
Then that it becomes noticable bigger, which especially for large blobs is a problem
Then that you always have to scan over this now larger large chunk to know it's end.
(Inserted of just skipping ahead).
Then that you have to encoded/decide it which might require some additional annotations depending on serialization library and can imply other annoyances.
The later point can also in some languages lead to you accidentally using a base64 string as bytes or the other way around.
Re:Base64 encoding and decoding at almost the speed of a mem...
by SlowRobotAhead on Wednesday November 06, 2019 @11:50AM
(#21464307)
JSON human readable is extremely nice. FWIW, I’ve found some binary formats like CBOR and MsgPack to be so almost human readable as binary when you are even slightly familiar with the format, or just convert almost instantly to readable JSON.
When I was developing a three component system, protos were not working for me. Too many schema changes and implementation differences to make the reduced binary size worth it. This was partly because the best protobuffer implementation in C (nanoPB) is just the hobby project of some dude, but mostly because coordinating schemas was annoying.
Re:Base64 encoding and decoding at almost the speed of a mem...
by SlowRobotAhead on Wednesday November 06, 2019 @11:45AM
(#21464262)
Protobuffs require sharing of a schema, then code generation for each language. Not ideal imo unless you really need the speed and the protos can be shared ahead of time.
Outside of Mongo, I haven’t seen BSON used anywhere.
CBOR however, is up and coming. Starting to look like it may be a first class citizen in AWS someday. On the IoT side they are already preferring it over JSON for Device Defender.
Re:Base64 encoding and decoding at almost the speed of a mem...
by fanf2 on Wednesday November 06, 2019 @06:06AM
(#21461415)
Base64 comes from the development of MIME in the early 1990s (see https://tools.ietf.org/html/rfc1341 section 5.2) though there are other similar encodings such as uuencode which predate it.
Re:Base64 encoding and decoding at almost the speed of a mem...
by guipsp on Wednesday November 06, 2019 @05:21AM
(#21461204)
I agree with the spirit and most of the content of this comment, except for the hard to random access part. It's not really hard to random access base 64.
Re:Base64 encoding and decoding at almost the speed of a mem...
by londons_explore on Wednesday November 06, 2019 @05:37AM
(#21461286)
Theoretically, yes it's easy.
Practically, base64 encode a video file and tell me how exactly you're going to allow the user to seek to any place they like within that video using only common libraries on common platforms... Theoretically easy, practically hard enough nobody does it.
Re:Base64 encoding and decoding at almost the speed of a mem...
by sitkack on Wednesday November 06, 2019 @01:18PM
(#21465280)
If the data being base64 encoded will also then be compressed, it is better to base16 encode (hex) and then apply compression. Base16 is faster to encode and compresses better (probably compresses faster, didn't test that).
2537012 Nov 6 10:10 test.tar.hex.gz
2954608 Nov 6 10:03 test.tar.b64.gz
All the caveats, test with your data, etc.
What would be nice is an email/http resilient compressed format that could be stuck inside of json strings that can be easily recovered.
Re:Base64 encoding and decoding at almost the speed of a mem...
by Asraelite on Wednesday November 06, 2019 @05:10PM
(#21468063)
That being true, it might be worthwhile for compression algorithms to detect base64 encoding (or any other base for that matter) and reinterpret it as base16 in order to improve compression rates.
base64 is 48.8% larger after compression on english text, whereas it would only be 33.3% larger without compression. The reason is because compression finds and eliminates repeating patterns, but base64 can make the same input data look totally different depending on which of the 4 input alignments it has.
Re:Base64 encoding and decoding at almost the speed of a mem...
by alboy on Wednesday November 06, 2019 @05:50AM
(#21461340)
Most compression algos operate on single-byte chunks of data and base64 encoding messes up the original byte alignment making the input appear more random than it is.
Re:Base64 encoding and decoding at almost the speed of a mem...
by MaxBarraclough on Monday November 11, 2019 @03:15PM
(#21508423)
I don't follow. An image compressor is guaranteed not to be given Base64 input.
A general-purpose compression scheme can easily detect when its input is Base64, so I don't see why Base64 should be particularly hard to compress, in principle at least.
Re:Base64 encoding and decoding at almost the speed of a mem...
by usr1106 on Wednesday November 06, 2019 @02:08AM
(#21460366)
Why is that an achievement? Isn't it so that during a memory copy the CPU is basically idling because memory is so much slower than the CPU? Weren't that hundreds of instructions per memory access? So instead of having it wait it can also do computations (just standard instructions). Or is base64 computationally so heavy that it cannot fit into the "gaps"? I certainly have not tried, just thinking from the high level view that CPUs are incredibly fast and main memory is incredibly slow in comparison. And of course assuming that the data does not fit into any cache.
Re:Base64 encoding and decoding at almost the speed of a mem...
by boomlinde on Wednesday November 06, 2019 @04:29AM
(#21460946)
> Why is that an achievement?
There are tables in the paper comparing its speed to other implementations. This is more than an order of magnitude faster than the implementation used in Chrome, and for small data more than twice as fast as their previous AVX2 based implementation (https://arxiv.org/abs/1704.00605). The paper even notes that in some cases, for large inputs, it's decoding base64 faster than you could memcpy the data for the simple reason that it needs only write 6/8ths of the bytes. In their tests, it's only when the data fits in the tiny L1 cache that memcpy is consistently somewhat faster (44 GB/s vs 42 GB/s). So this is not only fast when you have to wait for a slow memory access, but when the loads hit other caches.
Or are you asking in more of a rhetorical sense? Like, why hasn't this been achieved before, that it's obvious or something? AVX-512 is only some 3 years old and base64 is only one of many problems that could benefit from vectorized processing.
Re:Base64 encoding and decoding at almost the speed of a mem...
by vardump on Wednesday November 06, 2019 @02:16AM
(#21460395)
You're mostly right. It's often not that hard to completely saturate memory bus.
However, base64 has weird mappings that take some processing to undo in SIMD – can't use lookup tables and need to regroup bits from 8 bit to 6 bit width. That does take a lot of cycles without specialized bit manipulation instructions.
Also the data you'll need to process is often probably already hot in the CPU caches. Disk/network I/O can be DMA'd directly into L3 cache.
So I think this decoder is a useful contribution. But also somewhat no-brainer once you have those AVX-512 instructions available.
Re:Base64 encoding and decoding at almost the speed of a mem...
by mytailorisrich on Wednesday November 06, 2019 @03:39AM
(#21460706)
> However, base64 has weird mappings that take some processing to undo in SIMD – can't use lookup tables and need to regroup bits from 8 bit to 6 bit width. That does take a lot of cycles without specialized bit manipulation instructions.
Base64 uses lookup tables and the bit manipulations required are standard shifts and 'and', which are basic, fast instructions on any CPU. That seems exactly what they do here with an efficient use of AVX512 to make it fast(er).
Re:Base64 encoding and decoding at almost the speed of a mem...
by vardump on Wednesday November 06, 2019 @05:26AM
(#21461223)
Because table lookups don't vectorize. You could try to use vectorized gather (VPGATHERDD [0]), but so far in my experience it's been just as slow as using scalar code. (Then again, even gather instructions operate just on 32-bit data, so it wouldn't help anyways.)
So to perform multiple operations in parallel per core (SIMD), you'll have to write some code to perform the "lookup" transformation instead.
Re:Base64 encoding and decoding at almost the speed of a mem...
by bluesign on Wednesday November 06, 2019 @02:31AM
(#21460454)
I think you got it all wrong, memcpy comparison is given as a baseline.
It just shows even if you do not do anything (just copy the memory from place A to place B), you have some CPU usage baseline per GB. So this kind of saying: with this algorithm, you have very minimal CPU usage.
This algorithm is not doing excessive memory usage and Idling CPU because of memory bandwith, it is just doing very minimal CPU usage.
Re:Base64 encoding and decoding at almost the speed of a mem...
by useerup on Wednesday November 06, 2019 @04:23AM
(#21460920)
While it is correct that memory is orders og magnitudes slower than modern cpus, this is also why cpus sport multiple levels of caches.
memcpy operations (and base 64 encoding/decoding) are highly "local" operations. When the segment of memory they operate on has been placed in the cpu level1 or level2 caches, the operation is much faster and will not need to access main memory for each byte/word. Only when a memory barrier is encountered will the cpu need to flush/refresh to/from main memory.
Re:Base64 encoding and decoding at almost the speed of a mem...
by BubRoss on Wednesday November 06, 2019 @06:05AM
(#21461409)
You are thinking of random memory access latency, which is very slow. Reading memory in spans that are contiguous is very fast in comparison because of prefetching. Writing is the same to a certain degree because of the same efficiency with TLBs but is not as fast as prefetched reads.
Re:Base64 encoding and decoding at almost the speed of a mem...
by rurban on Wednesday November 06, 2019 @05:38AM
(#21461287)
Not yet usable asis. Normally AVX-512 instructions are subject to downclocking. Now they tried a very recent CPU which is not subject to downclocking anymore. They didn't add code to test for the CPU revision which stopped downclocking, and they didn't benchmark with those older CPU's. We can only guess the older AVX256 code is faster there, but not how much.
Re:Base64 encoding and decoding at almost the speed of a mem...
by wmu on Wednesday November 06, 2019 @11:26AM
(#21464092)
Speaking of benchmarking against older CPUs: the paper falls into the "short communication" category, we didn't want to discuss all possible hardware configurations.
Re:Base64 encoding and decoding at almost the speed of a mem...
by tomsmeding on Wednesday November 06, 2019 @04:04AM
(#21460822)
In the readme for Turbobase64, they state:
> Only the non portable AVX2 based libraries are faster
The paper in OP is about 2 times as fast as their previous AVX2 code. I think they are superior to Turbobase64 here (though with more stringent hardware requirements).
Re:Base64 encoding and decoding at almost the speed of a mem...
by eMSF on Wednesday November 06, 2019 @08:46AM
(#21462450)
Even 5% sounds sketchy indeed. However, AVX2 (or Haswell New Instructions) hasn't quite been on every Intel processor this decade, rather only since the 4th generation (late 2013), and only in Core-branded processors.
Re:Base64 encoding and decoding at almost the speed of a mem...
by bluesign on Wednesday November 06, 2019 @09:11AM
(#21462726)
I think 40GB/s result when it is not fitting L1 cache. If you check result graph, On L1 cache fitting data memcpy is ahead with big difference, then they are almost head to head
Re:Base64 encoding and decoding at almost the speed of a mem...
by mensetmanusman on Wednesday November 06, 2019 @07:53AM
(#21461978)
iOS 13 allows the Shortcuts app to encode/decode information as Base64 as part of the scripting language.
People are using this to do clever things like encoding a watermark image to overlay on a picture, because the Shortcuts app does not allow file attachments as part of the scripting language.
Re:Base64 encoding and decoding at almost the speed of a mem...
by vardump on Wednesday November 06, 2019 @02:09AM
(#21460372)
Now show me memory speed SIMD uint8_t histogram, and I'll be happy. Computing histogram is slow and it doesn't really vectorize. Yet it's required by many important algorithms like data compression.
Re:Base64 encoding and decoding at almost the speed of a mem...
by powturbo on Wednesday November 06, 2019 @06:07AM
(#21461417)
There is new AVX512 instruction (_mm512_conflict_epi32) supposed to solve this, but it can't make the histogram construction faster than the scalar functions.
Re:Base64 encoding and decoding at almost the speed of a mem...
by robocat on Wednesday November 06, 2019 @02:35AM
(#21460477)
Unfortunately using AVX512 instructions only gets a speedup in very specific situations and for many real world use cases it actually underperforms due to oddities of scaling and switching delays. Profiling for more than a few milliseconds is one place you see phantom gains, so take care not to be deceived.
Edit: not saying this isn't a true benefit here, just that claims of speed when using AVX512 need to be treated with fair scepticism for actual use cases.
Re:Base64 encoding and decoding at almost the speed of a mem...
by robocat on Wednesday November 06, 2019 @01:49PM
(#21465655)
I didn't realise that. My original post was just to warn that AVX512 benchmarks can be highly misleading. Everyone has troubles measuring AVX512 performance:
Re:Base64 encoding and decoding at almost the speed of a mem...
by rurban on Wednesday November 06, 2019 @05:40AM
(#21461296)
He is aware, but sidestepped these issues. so this code is only recommended on the newest Cannon Lake processors, but we really want to know for which CPU which method is best. What about AMD Rome e.g.?
Re:Base64 encoding and decoding at almost the speed of a mem...
by vagab0nd on Wednesday November 06, 2019 @08:16PM
(#21469491)
Question: with this kind of optimization, how does the program run different functions on different CPUs? Like the ones that don't support AVX512? Some kind of runtime dispatch based on CPU ID?
Re:Base64 encoding and decoding at almost the speed of a mem...
by wmu on Wednesday November 06, 2019 @11:18AM
(#21463995)
For instance our SSE/AVX2 algorithms have been included in a great, mature library written and maintained by Alfred Klomp: https://github.com/aklomp/base64 (the library includes also vectorized code for ARM CPUs).
Re:Base64 encoding and decoding at almost the speed of a mem...
by m0zg on Wednesday November 06, 2019 @02:32AM
(#21460460)
Many people don't realize this but today's memory is dramatically underpowered for today's CPU. Consider the following: you can, theoretically, get ~80GB/sec of memory bandwidth from a modern Intel CPU (assuming you use all channels, and there are no interrupts, etc). That's 20 billion floats or int32s per second. The same CPU can do ~2.3 fp32 TFLOPS. That's 115 ops for every fp32. And it's getting worse as more and more cores are put on the same memory bus.
For a flea check, consider that the chip has 18 cores and runs at ~4.2GHz. To achieve the theoretical throughput each core has to do ~30 single precision flops per cycle. Which sounds about right: HEDT intel chips can (theoretically) do 32 of those per cycle.
This, of course, ignores whatever throttling Intel would need to apply to maintain TDP, so a purely theoretical back of the envelope.
Re:Base64 encoding and decoding at almost the speed of a mem...
by satanspastaroll on Wednesday November 06, 2019 @03:43AM
(#21460723)
This should change soon, as EPYCs can do 204GB/s of memory throughput (plus tons of pcie4.0). They also aren't cpu tied, all of the SKUs get all of the lanes
Re:Base64 encoding and decoding at almost the speed of a mem...
by m0zg on Wednesday November 06, 2019 @04:58AM
(#21461102)
It's a NUMA though. You'll have to write software specifically for NUMA to get anywhere close to that number. To make matters worse, EPYC also doubles the number of cores. To be fair, though, it also has a substantial amount of cache, but cache is not a panacea if you need something in the main RAM. And if you're churning through gigabytes of stuff per second, you'll be needing that very, very often.
Re:Base64 encoding and decoding at almost the speed of a mem...
by gmueckl on Wednesday November 06, 2019 @09:43AM
(#21463026)
Yes, NUMA and large numbers of core will pose completely new challenges for software optimization. Applications will have to become aware of the memory architecture of the underlying machine. They will have to make explicit assignments of memory allocations and threads to NUMA nodes based on their domain specific needs. In some cases, even duplicating data structures may be the right call. This is going to challenge how most developers write fast software.
The other thing is that even seemingly trivial things like spawning and synchronizing with lots of threads will be much more complex on CPUs with many cores. At some point, naively looping over all threads is going to be too slow. I think that the limit is going to be around 64 cores. Past that, you should actually parallelize your worker thread management to stay efficient. There is precendent for this in HPC, e.g. MPI implementations.
Re:Base64 encoding and decoding at almost the speed of a mem...
by conradfr on Wednesday November 06, 2019 @08:29AM
(#21462280)
As an anecdote I have a Phoenix LiveView project where I've discovered that loading, converting and sending an (uncacheable) image as base64 via the websocket connection feels quicker and smoother (I didn't benchmark it) than only updating the src path and letting the browser load it (with http1, I would have to compare with http2).
Re:Base64 encoding and decoding at almost the speed of a mem...
by pantulis on Wednesday November 06, 2019 @04:29AM
(#21460948)
Well nothing's stopping you from making a living doing this kind of programming. I think the tech landscape is vibrant enough to have fun with it in many different ways!
Re:Base64 encoding and decoding at almost the speed of a mem...
by lifthrasiir on Wednesday November 06, 2019 @02:37AM
(#21460482)
Not only it is just a standard approach, it even misses a relatively common optimization for base64 decoding: instead of computing `(lut[a] << 6) | lut[b]` etc., one can precompute `lut6[x] = lut[x] << 6` and compute `lut6[a] | lut[b]` to avoid shifting. This optimization is famously used by Nick Galbreath's MODP_B64 decoder, which is used by Chromium [1] and turns out to be the most performant non-SIMD decoder according to Lemire et al. [2]
Note that not many Intel processors support the two specific CPU instructions used (vpermb and vpmultishiftqb).
Cannon Lake Core i3 (not many of these around due to delays with intel's 10nm fabrication)
Ice Lake Core i3, i5 and i7: Released in September 2019
Honestly, base64 has always been pretty fast for me, even large bitmaps >100kb. Compared to net latency it may be imperceptible to end user. But it still makes a terrific reference paper!
> In economics, a commodity is an economic good or service that has full or substantial fungibility: that is, the market treats instances of the good as equivalent or nearly so with no regard to who produced them.
Basically it's a common good, that is sold and purchased, with no other modifications required for this use-case. This commodity is currently low in supply, but is still a commodity.
I think hard drives, memory, power supplies are much closer to being fungible.
The problem start with knowing the encoding (base64, for small sizes sometimes hex, for one or two bytes sometimes arrays with all there ambiguities).
Then that you can't easily differentiate between a string a bytearray anymore (through you often shouldn't need to).
Then that it becomes noticable bigger, which especially for large blobs is a problem
Then that you always have to scan over this now larger large chunk to know it's end. (Inserted of just skipping ahead).
Then that you have to encoded/decide it which might require some additional annotations depending on serialization library and can imply other annoyances.
The later point can also in some languages lead to you accidentally using a base64 string as bytes or the other way around.
Well I guess that was enough ranting ;=)
JSON works where you need simplicity on the verge of being dumb, and human-readability.
When I was developing a three component system, protos were not working for me. Too many schema changes and implementation differences to make the reduced binary size worth it. This was partly because the best protobuffer implementation in C (nanoPB) is just the hobby project of some dude, but mostly because coordinating schemas was annoying.
Have you tried BSON or Protobufs to solve your annoyances? How did it go?
Outside of Mongo, I haven’t seen BSON used anywhere.
CBOR however, is up and coming. Starting to look like it may be a first class citizen in AWS someday. On the IoT side they are already preferring it over JSON for Device Defender.
Practically, base64 encode a video file and tell me how exactly you're going to allow the user to seek to any place they like within that video using only common libraries on common platforms... Theoretically easy, practically hard enough nobody does it.
2. Do whatever you said.
What would be nice is an email/http resilient compressed format that could be stuck inside of json strings that can be easily recovered.
A general-purpose compression scheme can easily detect when its input is Base64, so I don't see why Base64 should be particularly hard to compress, in principle at least.
There are tables in the paper comparing its speed to other implementations. This is more than an order of magnitude faster than the implementation used in Chrome, and for small data more than twice as fast as their previous AVX2 based implementation (https://arxiv.org/abs/1704.00605). The paper even notes that in some cases, for large inputs, it's decoding base64 faster than you could memcpy the data for the simple reason that it needs only write 6/8ths of the bytes. In their tests, it's only when the data fits in the tiny L1 cache that memcpy is consistently somewhat faster (44 GB/s vs 42 GB/s). So this is not only fast when you have to wait for a slow memory access, but when the loads hit other caches.
Or are you asking in more of a rhetorical sense? Like, why hasn't this been achieved before, that it's obvious or something? AVX-512 is only some 3 years old and base64 is only one of many problems that could benefit from vectorized processing.
However, base64 has weird mappings that take some processing to undo in SIMD – can't use lookup tables and need to regroup bits from 8 bit to 6 bit width. That does take a lot of cycles without specialized bit manipulation instructions.
Also the data you'll need to process is often probably already hot in the CPU caches. Disk/network I/O can be DMA'd directly into L3 cache.
So I think this decoder is a useful contribution. But also somewhat no-brainer once you have those AVX-512 instructions available.
Base64 uses lookup tables and the bit manipulations required are standard shifts and 'and', which are basic, fast instructions on any CPU. That seems exactly what they do here with an efficient use of AVX512 to make it fast(er).
This is doing exactly that with `vpermb` and `vpermi2b`.
Generally my point stands that LUTs are not vectorizable, except for small special cases like this.
So to perform multiple operations in parallel per core (SIMD), you'll have to write some code to perform the "lookup" transformation instead.
[0]: https://www.felixcloutier.com/x86/vpgatherdd:vpgatherqd
Yes and no. Memory has high latency, but memory bandwidth has stayed far more in line with CPU performance.
It just shows even if you do not do anything (just copy the memory from place A to place B), you have some CPU usage baseline per GB. So this kind of saying: with this algorithm, you have very minimal CPU usage.
This algorithm is not doing excessive memory usage and Idling CPU because of memory bandwith, it is just doing very minimal CPU usage.
memcpy with wide instructions has been an issue for libc.
current intel desktop processors have 'avx offset' which would suck if kicks on for mild base64.
overall wide instructions are nice and dandy and obliterate microbenchmarks but they may incur hidden costs, especially on low power cpus (laptops)
Compared to what, being content to suffering slower base64 decoding?
Why it's an achievement is obvious.
Your question should be rephrased better as "will this make much difference in most common scenarios"?
memcpy operations (and base 64 encoding/decoding) are highly "local" operations. When the segment of memory they operate on has been placed in the cpu level1 or level2 caches, the operation is much faster and will not need to access main memory for each byte/word. Only when a memory barrier is encountered will the cpu need to flush/refresh to/from main memory.
It is faster than SSE on Intel/AMD and SIMD NEON on ARM see benchmark at Turbo Base64: https://github.com/powturbo/TurboBase64
> Only the non portable AVX2 based libraries are faster
The paper in OP is about 2 times as fast as their previous AVX2 code. I think they are superior to Turbobase64 here (though with more stringent hardware requirements).
Not to say that Turbobase64 isn't impressive, but it's not at all the same level of performance.
It is impossible to have the speed of a good memcpy when you're copying 33% more like in base64.
People are using this to do clever things like encoding a watermark image to overlay on a picture, because the Shortcuts app does not allow file attachments as part of the scripting language.
Unfortunately, that's just not achievable with current x86 instruction set.
See
https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-us...
https://news.ycombinator.com/item?id=21029417
Edit: not saying this isn't a true benefit here, just that claims of speed when using AVX512 need to be treated with fair scepticism for actual use cases.
Considering that the author of that blog is one of the authors of this paper, I think he's very aware of the benchmarking issues.
"In GROMACS, transitions in and out of AVX-512 code can lead to differences in boost clocks which can impact performance. We are just going to point out the delta here." - from https://www.servethehome.com/intel-performance-strategy-team...
In my current desktop PC, it’s 700 GFlops theoretical compute versus 50GB/sec of theoretical RAM bandwidth.
For a flea check, consider that the chip has 18 cores and runs at ~4.2GHz. To achieve the theoretical throughput each core has to do ~30 single precision flops per cycle. Which sounds about right: HEDT intel chips can (theoretically) do 32 of those per cycle.
This, of course, ignores whatever throttling Intel would need to apply to maintain TDP, so a purely theoretical back of the envelope.
The other thing is that even seemingly trivial things like spawning and synchronizing with lots of threads will be much more complex on CPUs with many cores. At some point, naively looping over all threads is going to be too slow. I think that the limit is going to be around 64 cores. Past that, you should actually parallelize your worker thread management to stay efficient. There is precendent for this in HPC, e.g. MPI implementations.
That is what Superhuman uses for decoding base64 in browser.
[1] https://github.com/chromium/chromium/tree/master/third_party...
[2] https://github.com/lemire/fastbase64
The simple base64 scalar version is also faster the chromium implementation.