Re:Ask HN: Resources for General Purpose GPU development on ...
by morphle on Wednesday December 25, 2024 @08:41PM
(#42512551)
I forgot to add links to talk [5] by IBM Research on massively parallel Squeak Smalltalk and why it might be relevant for Apple Silicon reverse engineering and M4 clusters.
Talk [6] on free space optical interconnects without SerDes some day showing up on low power Apple Silicon (around M6-M8 models).
>do you happen to have some further information on that?
Yes, the information is in the detail custom documentation for the programmers and buyers of my Apple Silicon super computer. You can contact me for this highly technical report.
Do you know of people who might buy a super computer based on specification? Or even just buyers who will go for 'the lowest Capex and the lowest Apex supercomputer in 2025-227'?
Because the problem with HPC is that almost all funders and managers buy supercomputers with a safe brand name (Nvidia, AMD, Intel) at triple the cost and seldom from a super computer researcher as me. I have been designing, selling, programming and operating super computers since 1984 (I was 20 years old then), this M4 Apple Silicon Cluster will be my ninth supercomputer. I prefer to build them from the ground up with our own chip and wafer scale integration designs but when an off-the-shelf chip is good enough I'll sell that instead. Price/Performance/Watt is what counts, ease of programming is a secondary consideration for what performance you achieve. Alan Kay argues you should rewrite your software from scratch [2] and do your own hardware [3].
>Is it just a bunch of FPGAs issuing DMA TLPs?
No. The FPGA's are optional for when you want to flatten the inter-core (=inter-SRAM cache) networking with switches or routers to a shorter hop topology for the message passing like a Slim fly diameter two hop topology [4].
DMA (Direct Memory Access) TLPs (Transaction Layer Packets) are one of the worst ways of doing inter-core and inter-SRAM communication and on PCIe it has a huge 30% protocol overhead. Intel (and most other chip companies) can't design proper chips because they don't want to learn about software [2].
You should use pure message passing between any process, preferably in a programming language and a VM that uses pure message passing at the lowest level (Squeak, Erlang). Even better if you then map those software messages directly to message passing hardware as in my custom chips [3].
The reason to reverse Apple Silicon instructions for CPU, GPU and ANE are to be able to adapt my adaptive compiler to M4 chips but also to repurpose PCIe for low level message passing with much better performance and latency than DMA TLPs.
To conclude, if you want to get the cheapest CAPEX and OPEX M4 Mac mini supercomputer you need to rewrite your supercomputing software in a high level language and message passing system as the parallel Squeak Smalltalk VM [3].
[1] https://en.wikipedia.org/wiki/System_X_(supercomputer)
[2] https://www.youtube.com/watch?v=ubaX1Smg6pY
[3] https://vimeo.com/731037615
[4] https://www.youtube.com/watch?v=rLjMrIWHsxs