News for hackers. Stuff that matters.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @04:56PM (#41863779)
That's especially true because yours is a Xilinx FPGA. The one that they just attached to the latest gen mobile ryzens is 5x more capable too.

AMD are doing some fantastic work at the moment, they just don't seem to be shouting about it. This one is particularly interesting https://lore.kernel.org/lkml/DM6PR12MB3993D5ECA50B27682AEBE1...

edit: not an FPGA. TIL. :'(

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by errantspark on Wednesday October 16, 2024 @05:04PM (#41863852)
Wait sorry back up a bit here. I can buy a laptop that has a daughter FPGA in it? Does it have GPIO??? Are we seriously building hardware worth buying again in 2024? Do you have a link?
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by eightysixfour on Wednesday October 16, 2024 @05:21PM (#41863959)
It isn't as fun as you think - they are setup for specific use cases and quite small. Here's a link to the software page: https://ryzenai.docs.amd.com/en/latest/index.html

The teeny-tiny "NPU," which is actually an FPGA, is 10 TOPS.

Edit: I've been corrected, not an FPGA, just an IP block from Xilinx.

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by wtallis on Wednesday October 16, 2024 @05:30PM (#41864036)
It's not a FPGA. It's an NPU IP block from the Xilinx side of the company. It was presumably originally developed to be run on a Xilinx FPGA, but that doesn't mean AMD did the stupid thing and actually fabbed a FPGA fabric instead of properly synthesizing the design for their laptop ASIC. Xilinx involvement does not automatically mean it's an FPGA.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @05:40PM (#41864111)
Do you have any more reading on this? How come the XDNA drivers depend on Xilinx' XRT runtime?
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by wtallis on Wednesday October 16, 2024 @06:03PM (#41864296)
It would be surprising and strange if AMD didn't reuse the software framework they've already built for doing AI when that IP block is instantiated on an FPGA fabric rather than hardened in an ASIC.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @06:46PM (#41864630)
Well, I'm irrationally disappointed, but thanks. Appreciate the correction.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by almostgotcaught on Wednesday October 16, 2024 @05:55PM (#41864232)
because XRT has a plugin architecture: XRT<-shim plugin<-kernel driver. The shims register themselves with XRT. The XDNA driver repo houses both the shim and the kernel driver.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @06:43PM (#41864611)
Thanks, that makes sense.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by eightysixfour on Wednesday October 16, 2024 @05:33PM (#41864064)
Thanks for the correction, edited.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @05:33PM (#41864062)
Yes, the one on the ryzen 7000 chips like the 7840u isn't massive, but that's the last gen model. The one they've just released with the HX370 chip is estimated at 50 TOPS, which is better than Qualcomm's ARM flagship that this post is about. It's a fivefold improvement in a single generation, it's pretty exciting.

A̵n̵d̵ ̵i̵t̵'̵s̵ ̵a̵n̵ ̵F̵P̵G̵A̵ It's not an FPGA

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by almostgotcaught on Wednesday October 16, 2024 @05:56PM (#41864248)
> And it's an FPGA.

nope it's not.

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by boomskats on Wednesday October 16, 2024 @07:26PM (#41864925)
I've just ordered myself a jump to conclusions mat.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by almostgotcaught on Wednesday October 16, 2024 @07:51PM (#41865072)
Lol during grad school my advisor would frequently cut me off and try to jump to a conclusion, while I was explaining something technical often enough he was wrong. So I did really buy him one (off eBay or something). He wasn't pleased.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by dekhn on Wednesday October 16, 2024 @06:02PM (#41864293)
If you want GPIOs, you don't need (or want) an FPGA.

I don't know the details of your use case, but I work with low level hardware driven by GPIOs and after a bit of investigation, concluded that having direect GPIO access in a modern PC was not necessary or desirable compared to the alternatives.

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by errantspark on Thursday October 17, 2024 @12:18AM (#41866390)
I get a lot of use out of the PRUs on the BeagleboneBlack, I would absolutely get use out of an FPGA in a laptop.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by dekhn on Thursday October 17, 2024 @12:49AM (#41866503)
It makes more sense to me to just use the BeagleboneBlack in concert with the FPGA. Unless you have highly specific compute or data movement needs that can't be satisfied over a USB serial link. If you have those needs, and you need a laptop, I guess an FPGA makes sense but that's a teeny market.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by pclmulqdq on Wednesday October 16, 2024 @05:32PM (#41864048)
It's not an FPGA. It's a VLIW DSP that Xilinx built to go into an FPGA-SoC to help run ML models.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by almostgotcaught on Wednesday October 16, 2024 @05:56PM (#41864242)
this is the correct answer. one of the compilers for this DSP is https://github.com/Xilinx/llvm-aie.
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by beeflet on Wednesday October 16, 2024 @05:08PM (#41863876)
It would be cool if most PCs had a general purpose FPGA that could be repurposed by the operating system. For example you could use it as a security processor like a TPM or as a bootrom, or you could repurpose it for DSP or something.

It just seems like this would be better in terms of firmware/security/bootloading because you would be more able to fix it if an exploit gets discovered, and it would be leaner because different operating systems can implement their own stuff (for example linux might not want pluton in-chip security, windows might not want coreboot or linux-based boot, bare metal applications can have much simpler boot).

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by walterbell on Wednesday October 16, 2024 @06:44PM (#41864617)
Xilinx Artix 7-series PicoEVB fits in M.2 wifi slot and has an OSS toolchain, http://www.enjoy-digital.fr/
Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by numpad0 on Wednesday October 16, 2024 @06:23PM (#41864435)
Sorry for an OT comment but what is going on with that ascii art!? The content fits within 80 columns just fine[1], is it GPT generated?

1: https://pastebin.com/raw/R9BrqETR

Re:AI PCs Aren't Good at AI: The CPU Beats the NPU by davemp on Wednesday October 16, 2024 @09:58PM (#41865733)
Unfortunately FPGA fabric is ~2x less power efficient than equivalent ASIC logic at the same clock speeds last time I checked. So implementing general purpose logic on an FPGA is not usually the right option even if you don’t care about FMAX or transistor counts.