News for hackers. Stuff that matters.

jgr0007

Re:You Could Just Choose Optimism on Saturday May 03, 2025 @12:30AM (#43876863)
"I wish there were more good role models of this behavior. Or maybe there are, and I’m too much of a grouchy complainer to see them. Who comes to mind for you?"

Immediately thought of, and could not stop thinking of the Calvin & Hobbes camping trip. https://static.wikia.nocookie.net/candh/images/3/30/8-16.gif...

Thanks for posting this view: I had read the "Land Value Tax"-side of the argument before (and somewhat agree with it), but I hadn't seen someone post good reasons _not_ to support LVT. I think your reply - and your other replies - are pretty good reasons
Re:LLM in a Flash: Efficient LLM Inference with Limited Memory on Thursday December 21, 2023 @05:03PM (#38727678)
I was hoping to find some sort of "how this feature will be exposed to users" section in the conclusion, but maybe that discussion is out of scope. Does this kind of feature then bubble up into CoreML as API calls and settings, where you need to set, for example, a _use_flash_ flag? Or does does this just become a runtime optimization opaque to the user? Curious if anyone knows of good talks/presentations where Apple discusses their CoreML, Metal, etc development roadmaps.
Re:Qualcomm’s Hexagon DSP, and Now, NPU on Thursday October 05, 2023 @12:54AM (#37775025)
> However, the Hexagon DSPs are notoriously hard to program.

I disagree the Hexagon is "hard" to program - the HVX intrinsics are fairly straightforward if you've used neon/avx/etc. The problem is that you don't get access to the HMX/HTA/(whatever they call the NPU currently), just HVX. So for running on DSP your options are (1.) Qualcomm's Hexagon SDK which doesn't give you access to the tensor hardware, and (2.) their SNPE SDK, which actually has access to HMX/HTA/etc, but doesn't let you program against the hardware directly. Instead, SNPE is supposed to convert your caffe/tf/onyx/torch models to Qualcomm proprietary format so it can map ops to the most appropriate hardware. What actually happens is that their conversion tools fall apart the first time you don't use one of their AlexNet/ResNet vanilla examples and try to convert a prod-grade model. Combine the brittleness of their conversion tools with the lack of documentation/support and it becomes impossible to use the hardware.

All that being said, would be interested to hear if anyone has had luck with other means of using their (admittedly impressive) hardware. Maybe a better approach would be to try using Halide for Hexagon and OpenCl for Adreno? They were also going to release a QNN update this year or next year - not sure if they've allowed anyone access to it. Its really a shame that the hardware is so good, but the software is so clunky.