News for hackers. Stuff that matters.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by com2kid on Tuesday April 14, 2020 @05:22PM (#22871616)
The 7F52 has 16MB of cache per core.

I'd love to see what is possible with a tiny runtime/OS in the kilobyte size and running a microservice written in a native language off of each core, everything out of the L3 cache.

I imagine the throughput would be amazing. Single thread per core, e.g. cooperative multitasking. Do this for stream orientated workflows, or even for processing data that is in reasonable sized chunks, it might be screaming fast!

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by yjftsjthsd-h on Tuesday April 14, 2020 @06:06PM (#22872076)
When I was younger, I was very pleased when I stumbled across the trick of putting all of a DOS system on a RAM-disk, which made the system fly. Then I got to college and discovered in my CS classes that CPUs now had multi-megabyte on-CPU cache, which immediately led me to wonder how hard it would be to make a DOS system that ran completely out of cache. I'm not convinced that it would be practical for any number of reasons, but it still blows me away every time I look at what modern CPUs have in-package:)
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by abbeyj on Tuesday April 14, 2020 @07:21PM (#22872650)
Coreboot does something similar to this. Very early in the boot process the RAM has not yet been initialized and so cannot be used. You need to execute some code to set up the RAM before you can use it. One approach to handling this is to write that initialization code in assembly language and make sure it only uses registers and doesn't touch any memory.

But that's inconvenient. You'd much rather be able to write your code in something like C. Any C code compiled by an ordinary compiler is going to access memory (at least for the stack). Writing and maintaining a custom C compiler for this is a lot of work and still comes with a lot of limitations on the C code that it will accept.

So they set up the cache in a way that it will never try to write its contents out to RAM. Then you can use the output of an ordinary C compiler that uses memory accesses and those accesses will all be served by the cache and never touch RAM. They call this "Cache as RAM".

With some work you could probably boot into DOS like this.

https://www.coreboot.org/data/yhlu/cache_as_ram_lb_09142006....

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by als0 on Tuesday April 14, 2020 @07:31PM (#22872725)
Not just coreboot, pretty much any BIOS uses cache-as-RAM.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by ahmedalsudani on Tuesday April 14, 2020 @10:05PM (#22873710)
Wow that is so cool. Most of us never have to dig that deep and so weird stuff like that usually flies completely under the radar...

Thank you for sharing!

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jnwatson on Wednesday April 15, 2020 @03:04AM (#22875301)
This is a common approach for most bootloaders, in fact for several microprocessors (the IBM 440 comes to mind), it is the only way to boot.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by myself248 on Wednesday April 15, 2020 @09:52AM (#22877684)
Is there a way to mess with the cache or MMU and prevent it from _trying_ to write out to DRAM, and be able run a minimal system purely from cache with the DRAM slots completely empty?
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by mcot2 on Wednesday April 15, 2020 @02:57AM (#22875268)
Interesting that no one to my knowledge has built an architecture where sram is ‘memory’ such that it would be transparent to C code but no slower dram is used. 32MB is more than my 1998 computer had.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jnwatson on Wednesday April 15, 2020 @03:03AM (#22875289)
Many embedded systems have a block of SRAM directly addressable.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by rijoja on Wednesday April 15, 2020 @03:02AM (#22875285)
modify tcc maybe
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by keanebean86 on Tuesday April 14, 2020 @06:28PM (#22872262)
Run windows 95 in L3 cache and use this to store the install: https://github.com/prsyahmi/GpuRamDrive
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jschwartzi on Tuesday April 14, 2020 @06:47PM (#22872415)
This is effectively what you do with some of the smaller MCUs -- you're running out of a megabyte or two of on-die static RAM. This is more to eliminate latency in RTOSes than it is to improve throughput but it's the same idea.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by gpderetta on Tuesday April 14, 2020 @08:32PM (#22873119)
More than eliminating latency I guess is done to eliminate latency variance, which is inherent in the relatively unpredictable behaviour of caches, and is required to characterize worst case scheduling of realtime systems.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by yetihehe on Wednesday April 15, 2020 @04:24AM (#22875643)
Megabyte of RAM? That's a very big MCU for most of embedded programmers. 20KB is plenty for most, and some even go to zero RAM (ATtiny11).
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by vardump on Wednesday April 15, 2020 @06:48AM (#22876305)
Working with embedded systems. Different MCUs' memory ranges from bytes to gigabytes. What's plenty for one project isn't for another.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jschwartzi on Wednesday April 15, 2020 @06:14PM (#22883721)
Yeah I've worked on projects with Toshiba MCUs with 10's of kB of RAM, and the last truly embedded project I was on used a safety-critical MCU from TI with about a megabyte of internal ECC RAM.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by baybal2 on Tuesday April 14, 2020 @08:42PM (#22873181)
> I'm not convinced that it would be practical for any number of reasons, but it still blows me away every time I look at what modern CPUs have in-package:)

The optimism is premature. A CPU may flush cache even when it pages exactly the same page. Modern CPUs have too much of completely unfathomable prefetch, and cache coherence logic

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by londons_explore on Wednesday April 15, 2020 @04:52AM (#22875751)
But they're also configurable via various config registers, and can be set to not touch RAM, as most early-boot firmware uses.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by baybal2 on Wednesday April 15, 2020 @12:42PM (#22879872)
No, all tricks to control cache behaviour are not 100% reliable on modern CPUs, or at least intel ones
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by sdenton4 on Wednesday April 15, 2020 @03:50AM (#22875515)
Ha. I still use ram disk for local training of machine learning models. You can get a SOLID throughput boost by just loading up the dataset in RAM... My work desktop has something like 192Gb, so can handle a pretty sizeable dataset this way.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by thesmileyone on Wednesday April 15, 2020 @06:57AM (#22876349)
Isn't this what 5-eyes have been doing with MiniX3? etc
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by ChuckMcM on Tuesday April 14, 2020 @05:36PM (#22871766)
There is an interesting argument to be made for DSP like workloads for these chips. With x8 PCIe 4.0 lanes to carry inter-socket traffic.

If I had infinite money to spend I would build a hypercube out of these bad boys. each "atom" having 6 x8 PCIe connectors (up,down,left,right,front,back) with corner nodes dedicating a x16 PCIe port for I/O in and out of the cube.

Immerse that bad boy in flourinert and contemplates the deepest secrets of the universe :-)

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by BeeOnRope on Tuesday April 14, 2020 @05:44PM (#22871861)
That would just be a cube, right?
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by ChuckMcM on Tuesday April 14, 2020 @06:00PM (#22872027)
Nominally hypercube architecture is 'n' dimensional[1], which relates to the number of channels at each vertex. Assuming you took the front and back plane of this thing and connected them together it would qualify as having 6 dimensions (6 channels per vertex). The first such system I had a chance to play with was the Intel one[2] which was order 6 using Intels iAPX432 processors.

[1] The (original?) Oak Ridge National Labs paper -- https://www.osti.gov/biblio/6487986-parallel-computing-hyper...

[2] https://www.nature.com/articles/313616b0.pdf?proof=trueIn

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by Dylan16807 on Tuesday April 14, 2020 @08:55PM (#22873275)
Oh, I see. If your cube is exactly 4x4x4, and you loop it in each dimension, then it's topologically equivalent to an order-6 hypercube with a node on each vertex.

In the general case, you need log2(N) connections in each direction to turn an NxN or NxNxN topology into a hypercube, so it's not as simple as "left/right". (For example, for N=8, you might connect each position to position xor 1, position xor 2, and position xor 4.)

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by dr_zoidberg on Tuesday April 14, 2020 @05:52PM (#22871954)
OP's referring to an old architecture which was called "the hypercube" which involved a couple (I can't remember if 16 32, 64 or what number) of microprocessors connected to one another in a certainly-not-2d topology.

Edit: oh a bit of searching and it was in the order of thousands of microprocessors. It was called "Connection Machine", you can read about it in Wikipedia[0].

[0] https://en.wikipedia.org/wiki/Connection_Machine

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by josh2600 on Tuesday April 14, 2020 @05:57PM (#22872009)
One of my favorite essays ever is on the subject of the Connection Machine and Richard Feynman: http://longnow.org/essays/richard-feynman-connection-machine...
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by buildbot on Tuesday April 14, 2020 @07:53PM (#22872879)
Thank you for this, that was a really excellent read. Nothing better than old computer architecture and some early ML!
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by kick on Tuesday April 14, 2020 @11:27PM (#22874189)
TMC was one of the coolest companies of all time; I really wish they were talked about more often.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by Paul-ish on Tuesday April 14, 2020 @07:48PM (#22872840)
I learned from an electrical engineer that works on systems for the power grid that running the whole OS in CPU cache is a requirement for some of those systems. I guess that makes sense when you are working with things that move at the speed of electricity.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by person_of_color on Tuesday April 14, 2020 @10:14PM (#22873759)
No, the system is probably using tightly coupled memory; the CPU has single cycle access to all of system memory, effectively making a cache redundant.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by londons_explore on Wednesday April 15, 2020 @04:56AM (#22875774)
And the reason is likely "we are building something realtime, we can't afford variable latency".

For the electricity grid in today's world, that's over-engineering things. The 60Hz electricity grid doesn't really need anything being changed at more than 600Hz, and considering your CPU core runs at 3,000,000,000Hz, you can totally waste a lot of cycles before you start missing deadlines.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jstrong on Wednesday April 15, 2020 @10:29AM (#22878113)
"This meeting is to announce we've settled on an architecture for the next gen grid optimization program: Electron UI with a Ruby on Rails app underneath the hood that calls to AWS Lambda for anything that's cpu-heavy. State is persisted to Firebase as serialized YAML, which should scale really well. And to our normal crowd of performance zealots, don't even start with your normal "latency" and "outages" griping. We're comfortable with this stack and besides, we can waste a _lot_ of cycles before we start missing deadlines!"
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by HelloNurse on Wednesday April 15, 2020 @06:13AM (#22876143)
Even if your CPU cycle budget is large, designing a realtime system requires proving thet it won't be exceeded, by adding up the worst-case running time of the worst-case sequence of events. Avoiding high latency memory gives a few orders of magnitude of margin, which could be needed to afford good functionality on a reasonably cheap CPU; it's not a mere useless improvement from "fast enough" to "as fast as possible".
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by diroussel on Wednesday April 15, 2020 @02:18PM (#22881124)
In hard realtime systems, the problem is not "we have enough time to do the computation" but "we can prove the computation will run in the available time".

The complexity in modern mulit user operating systems (many processes, a run queue, spin locks) along with modern processors (multi tier memory architecture, micro-code, branch prediction, long stalls for memory access) mean it's very hard to prove that anything will run in finite time.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by SAI_Peregrinus on Thursday April 16, 2020 @09:17AM (#22889010)
IMO the terms "hard realtime" and "soft realtime" are unnecessarily confusing to beginners. I think that "deterministically bounded latency" and "probabilistically bounded latency" are more descriptive. But they're long and difficult to type, so we'd inevitably call them DBL and PBL, and be back at confusion.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by dx034 on Wednesday April 15, 2020 @07:15AM (#22876453)
And yet, so much software on modern CPUs takes seconds rather than milliseconds to perform and output simple calculations. Especially with the mantra "developer time is more expensive than computing power", few developers nowadays have recent experience of optimizing programs for sub-ms reaction times. You could probably make it work with standard hardware. But since developers have to be retrained for that field anyway, why not use hardware that's battle-tested and gives you more leeway?

In addition, the advantage of optimizing at every point is that you can use hardware not designed at the limit of what's possible, likely giving you lower error rates.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by rckoepke on Tuesday April 14, 2020 @11:14PM (#22874114)
Would KolibriOS fit the constraints to do something like this? Per the description it "requires only a few megabyte disk space and 8MB of RAM to run." but has a recognizably usable GUI and office suite.

https://kolibrios.org/en/

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by rurban on Wednesday April 15, 2020 @04:57AM (#22875786)
Better the original MenuetOS which supports now 64bit, though this is not open source anymore.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by lallysingh on Tuesday April 14, 2020 @07:29PM (#22872712)
Unikernel as a c++ header would be a good candidate: https://github.com/includeos/IncludeOS
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by dx034 on Wednesday April 15, 2020 @07:17AM (#22876472)
kdb+ [1] used to advertise already about a decade ago that their database code fits into L3 cache, making it extremely fast in execution. With 16MB, you can probably fit the core of a pretty much fully featured database in there, accelerating data driven jobs that GPUs are suited for.

[1] https://kx.com/

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by tomlu on Tuesday April 14, 2020 @08:59PM (#22873315)
This is kind of what the SPUs of the PS3 were like, except with way way less RAM.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by SkyPuncher on Tuesday April 14, 2020 @05:55PM (#22871992)
Besides cryptographic heavy services, I'm having trouble imagining a microservice that would benefit from this.

Disk, Network, or in-memory artifact reference would seem to be much bigger bottlenecks.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by com2kid on Tuesday April 14, 2020 @06:27PM (#22872252)
I'm thinking of simplicity. No VMs, no containers, just a single binary.

The majority of microservices I've written would fit into a system like this just fine.

NodeJS is built around the idea that a single thread of execution is enough for a stupidly large % of backend code. On top of that is ton of infrastructure to deploy slews of 100 line JS microservices that each do one thing.

So, back to basics. No VM, no disk at runtime, a single statically linked binary running on top of nearly bare metal.

Available memory is whatever space the program code doesn't take up.

Ignoring l3 being unaddressable, I wonder if UEFI applications are powerful enough to pull something like image deployment off.

This is probably all less efficient than the current Russian doll system of VMs and containers in use right now, but it is fun to imagine!

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by londons_explore on Wednesday April 15, 2020 @04:58AM (#22875796)
> Ignoring l3 being unaddressable

If you set the right config registers, you can map the cache to a linear address space and use it like RAM.

I think to do that, you loose access to actual RAM though, so it isn't awfully practical.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by wmf on Tuesday April 14, 2020 @06:33PM (#22872306)
Basically unikernels on the Jailhouse hypervisor.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by ajflores1604 on Tuesday April 14, 2020 @09:54PM (#22873657)
I can see this being used in hft houses
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by dx034 on Wednesday April 15, 2020 @07:19AM (#22876480)
I thought they all use custom hardware by now? But it's probably useful for applications that go beyond simple algorithms, work on time series data could be much faster.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by smabie on Thursday April 16, 2020 @12:35PM (#22891089)
They use custom hardware for the basic kind of latency arb but software is used for more complicated stuff.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by salawat on Tuesday April 14, 2020 @11:24PM (#22874165)
My eyes read that as

>I can see this being used to lift houses.

My evening has been made that much more interesting.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by jfkebwjsbx on Tuesday April 14, 2020 @08:34PM (#22873130)
I don't understand why you would want to run tiny systems for that. Properly configured Linux processes will do the same.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by bgirard on Tuesday April 14, 2020 @05:34PM (#22871754)
I'm not sure with cache aliasing you'd be guaranteed that it would stay all in L3.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by com2kid on Tuesday April 14, 2020 @06:15PM (#22872147)
It's have to be a special setup for sure, a board with no memory even hanging off of it.

In theory all you'd need is some tight integration with the NIC (or a few of em!) and tiny driver that knows how to DMA to the NIC.

Larger problem of course is that L3 isn't addressable. I wonder how much money a cloud provider would have to pony up to AMD to get an exception to that? :)

A huge # of workloads would fit in 16MB with no storage hanging off. Heck I think every microservice I've ever written could manage that.

There have been some efforts of late to make a cloud first OS, honestly once you remove everything except for "talk to network controller" and "parse JSON", well so long as you don't mind writing in a language that isn't JS, a few MB is plenty for lots of workloads.

Re:AMD Claims World’s Fastest Per-Core Performance with New ... by gpderetta on Tuesday April 14, 2020 @08:36PM (#22873142)
Apparently there are ways to treat the cache as a static scratch pad. It is used by BIOSes during early startup, but I'm not sure whether the magic incantations are publicly known. It is second hand knowledge though, so it might just be folklore.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by Dylan16807 on Tuesday April 14, 2020 @09:12PM (#22873411)
If you allocate one big block it would fit even in a 1-way cache. Modern chips appear to have 16-way L3, so you'd have to be trying to spill.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by ashtonkem on Wednesday April 15, 2020 @03:37AM (#22875458)
For most general purpose work, you’d probably lose throughout. Chances are you’d rather have the heap be in the cache rather than the code segments.
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by starpilot on Tuesday April 14, 2020 @06:08PM (#22872095)
kdb
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by smabie on Thursday April 16, 2020 @12:36PM (#22891102)
I'm not sure why you are getting downvoted, because kdb+/q would be a perfect candidate. Too bad kOS is no longer still a thing (or maybe it never was).
Re:AMD Claims World’s Fastest Per-Core Performance with New ... by lucio on Wednesday April 15, 2020 @03:31AM (#22875428)
>640k ought to be enough for anybody