It depends on the workload. We tried the ampere emag, and what killed it for us was that TLS performance was nowhere near modern x86-64 CPUs (Intel or AMD)

Was that heavily cipher dependent? I wouldn't be surprised if Chacha20 performed much better than AES w/o any hardware acceleration (other than SIMD instructions)

Ah, that's interesting. We don't use chacha20. This was AES-GCM

The situation is probably better now that ARMv8 has some crypto-specific instructions, but AES-GCM on older ARMs performed awfully without the instructions specifically for doing AES and Galois field multiplication

As a n00b, is that down to poor or lacking encryption hardware support ala AES-NI? Or something else?

Yes.

You did make sure the AES instructions were used, right? I wouldn't be surprised the AES unit on the eMAG is relatively slow — other units seem to be as well, e.g. in my silly CRC32 benchmark, the Arm Cortex-A72 did 1kb in 79 ns at just 2.0 GHz while the eMAG did it in 103 ns at much faster clocks (3.0 or 3.3 GHz).

I suspect they might have reused these HW blocks from the old Applied Micro X-Gene :D

But now on the new product, it's all Arm Neoverse cores, it's gonna be great.

did you use them on Packet?