rnrn
Lots of emphasis on no kernels written and nothing recompiled but the intro and conclusion somehow suggest this is also the value of runinfra’s agents the rewrite software ?
Walk through the theory of peak decode throughout based on memory bandwidth with one weight load pass per token to provide a target for optimized decide but then doesn’t address low bandwidth utilization in that scenario at all and uses speculative decode where that theory doesn’t apply
This is not an accurate summary of what they developed.
Using .eh_frame to unwind stacks without frame pointers is not novel - it is exactly what it is for and perf has had an implementation doing it since ~2010. The problem is the kernel support for this was repeatedly rejected so the kernel samples kilobytes of stack and then userspace does the unwind
What they developed is an implementation of unwinding from an eBPF program running in the kernel using data from eh_frame.
HP semiconductors went HP -> Agilent -> Avago, now broadcom.
The activated experts is only available after routing, at which point you need the weights immediately and will have very poor performance if they are across PCIe
Also as other commenters have mentioned, redirecting allocations to managed memory would also enable similar oversubscription
And the hijack approach only makes sense for making apps have this behavior with no changes, and could be done with minor app changes (e.g. PyTorch has a pluggable allocator interface). App changes also enable intentionally placing specific allocations.
My impression is that this is vibe from beginning to end, starting from a design that only makes sense if you are hallucinating
Regardless of whether it would be better in some situations to align to 128 bytes, 64 bytes really is the cache line size on all common x86 cpus and it is a good idea to avoid threads modifying the same cacheline.
I don’t think it is accurate that Intel CPUs use 2 cache lines / 128 bytes as the coherency protocol granule.
Yes, there can be additional destructive interference effects at that granularity, but that’s due to prefetching (of two cachelines with coherency managed independently) rather than having coherency operating on one 128 byte granule
AFAIK 64 bytes is still the correct granule for avoiding false sharing, with two cores modifying two consecutive cachelines having way less destructive interference than two cores modifying one cacheline.
> It is my Great Honor to report that the United States of America now fully owns and controls 10% of INTEL, a Great American Company that has an even more incredible future. I negotiated this Deal with Lip-Bu Tan, the Highly Respected Chief Executive Officer of the Company. The United States paid nothing for these Shares, and the Shares are now valued at approximately $11 Billion Dollars. This is a great Deal for America and, also, a great Deal for INTEL. Building leading edge Semiconductors and Chips, which is what INTEL does, is fundamental to the future of our Nation. MAKE AMERICA GREAT AGAIN! Thank you for your attention to this matter.
“The United States paid nothing for these Shares”
https://truthsocial.com/@realDonaldTrump/posts/1150744446179...
> The Department of Commerce is authorized to provide funding in various forms, including grants, cooperative agreements, loans, and loan guarantees, in exercising its Section 9902 authorities
https://www.congress.gov/crs-product/R47523
AFAICT the relevant section of law says it is up to the Secretary of Commerce to determine the funding type to be used for the semiconductor financial assistance
https://www.congress.gov/116/plaws/publ283/PLAW-116publ283.p...
See my other comment : https://news.ycombinator.com/item?id=44995799
That they were grants instead of any other instrument appears to be a Biden Commerce decision, not a congressional one.
I’m no lawyer and could certainly be missing something i the law that says it has to be grants but from what I see it looks like figuring out what to do the the money was pretty much delegated to dept of commerce with limited direction about eligibility and review criteria.
I don't think that makes it obvious at all, given that the none of these invented devices actually worked, and the first working MOSFETs weren't until the late 50s after a research program of a few additional decades by a bunch of solid-state physicists at Bell Labs (who did know and develop quantum theories of solids - Shockley, Bardeen, Brattain - not successful in making a FET -Atalla, Kahng, many others)
"Electrons and Holes in Semiconductors" was published almost a decade before any functional MOSFET was constructed.
> For designing MOSFETs, you just need to use classical electrodynamics, together with several functions that provide the semiconductor material characteristics, like intrinsic free carrier concentration as a function of temperature, carrier mobilities as functions of temperature and impurity concentrations (and electric field at high fields), ionization probabilities for impurities, avalanche ionization coefficients, dielectric constants, and a few others.
It sounds like you are describing what's required to parameterize some of the traditional semi-classical models of MOSFETs and understand the operating principles at that level.
but FETs work by bending the energy levels of the conduction band so there needs to be a band to bend, and if there's no band gap at the fermi level you can't have a FET, which makes it seem pretty dependent on quantum effects to me even without going deeper than necessary to understand how it can work.
Maybe one could have been engineered with no idea why silicon has the special material properties that it does and why doping changes those properties but AFAIK it never was, and being able to explain and understand band structure seems pretty important to build a working device.
Forming a conductive channel in silicon in any FET and semiconductivity in general is an inherently quantum effect too, right?
I don’t think Illinois has actually awarded any of that funding to providers to build anything yet. It looks like the original schedule was to start awarding grants this summer after planning process from 2021-2025.
maybe some vendors have had an equivalent to uniform registers for 20 years, but per the articles’ references they are new in nvidia GPUs in turing (2018)
AMD continuing to have the Xilinx line of FPGAs with some Cortex cores and having Arm management cores that run firmware doesn’t really address this.
in theory only.
caravel had hold time violations and the pin configuration mostly didn’t work for the first 5 or 6 sponsored OpenMPW shuttles.
> Furthermore, we find the system could also find other novel exploits in the benchmark’s tasks
“Novel exploit” is a pretty fancy and generous way of saying that some of the kernels wrote a constant value to the entire output because the evaluation code only tested one set of inputs that can pass if you replace the computation with a memset.
As I write this, https://pub.sakana.ai/ai-cuda-engineer/kernel/2/23/optimize-... is on their top of their list of speedups, with a claim of 128x speed up on a fused 3D convolution + groupnorm + mean.
The generated implementation doesn’t do a convolution.
The author has updated the post with corrected AVX measurements, with the original ~340 GB/s revised down to 31.7 GB/s. (Thanks CowFreedom)
The post is about dot product, not matrix multiply. Dot product has no data reuse