fpgamlirfanboy

edit:

> They are working on 2-3 CPU designs, 2-3 AI accelerators, all the IO stuff, etc

also do we actually know how much they're designing wholecloth and not buying off the shelf?

edit2: i think Tenstorrent is great and i'll probably buy a devkit soon (as well as maybe apply lol...)

of course https://github.com/pytorch/pytorch/blob/main/torch/utils/_py...

> Wigand turned his attention to improving tobacco additives, some of which were designed for "impact boosting", using chemicals like ammonia to enhance absorption of nicotine in the lungs and affect the brain and central nervous system faster. Wigand believed this process was a deliberate attempt to increase addiction to cigarettes.

i work on compilers+hardware so that's the backdrop here.

here's a real hypothetical for you (ie it happened but i'm not going to use specifics): our internal proprietary compiler is spitting out incorrect atomics instructions that deadlock our internal proprietary multi-core DNN accelerator. this incorrect code is downstream of a big, lucrative, customer's (you know which one) LLM model.

now the question: is it the "passion coder" that will solve this or the grinder?

It's amazing how many people in tech can't imagine that the same people that can grind med school or IB or big4 accounting or white-shoe law somehow can't grind the same way in tech. Newsflash: the majority of people in FAANG are grinders not "passion coders".

Personally I hate this job but I'm very good at it and it was either law school with my 98% LSAT or tech. I picked tech because reading and writing briefs all day seemed somehow worse.

Are people getting paid to repeat this ad nauseum?

they're not the same - versal acaps (whatever you want to call them) have AIE1 arch while phoenix has AIE2 arch. there are significant differences between the two arches (local memory, bfloat16, etc.)

> Microsoft won't ever be able ... their business model is ... have had enough.

statements made with complete certitude, as if these things aren't nebulous to begin with or in the process of being decided. it reads like the opening line to a christopher nolan superhero movie (i.e., a call to arms) rather than a position or even an opinion.

i personally don't understand what people get from the act of writing/posting such things - i mean i'd get it if we were on some hill about to rush into battle but this is an anonymous/faceless tech news forum.

i never said i hated yosys/nextpnr? i said somewhere that yosys makes the uber strange decision to use C++ as effectively a scripting language ie gluing and scheduling "passes" together - like they seemed to make the firm decision to diverge from tcl but diverged into absurd territory. i wish yosys were great because it's open source and then i could solve my own problems as they occurred. but it's not great and i doubt it ever will be because building logic synthesis, techmapping, timing analysis, place and route, etc. is just too many extremely hard problems for OSS.

all the vendor tools suck. it's just a fact that both big fpga manufacturers have completely shit software devs working on those tools. the only tools that i've heard are decent are the very expensive suites from cadence/siemenns/synopsis but i have yet to be in a place that has licenses (neither school nor day job - at least not in my team). and mind you, you will still need to feed the RTL or netlist or whatever that those tools generate into vivado (so you're still fucked).

so i don't have advice for you on RTL - i moved one level up (ISA, compilers, etc.) primarily because i could not effectively learn by myself i.e., without going to "apprentice" under someone that just has enough experience to navigate around the potholes (because fundamentally if that's what it takes to learn then you're basically working on ineluctably entrenched tech).

>please show me the source line number in pytorch or tensorflow that defines this number

why? because the original comment makes a claim about pytorch. it's all right there in black and white.

because they have all of the gee-whiz factor of a freshman calc proof of the chain rule that divides and multiplies infinitesmals and absolutely not enough of the substance necessary to prove much more than that. they are absolutely, in the research literature, at best an anachronism (harkening back to leibniz) and at worst a parlor trick.

in literally my first response i provided the most trivial counter-example to the magic of non-standard analysis. no answers (crickets). i surmise this is because the people in here talking it up aren't really serious.

> You're right on one count: no library implements forward mode

there are plenty of CFD type libraries that implement forward mode. they also do not use what i'm calling "magical" dual numbers i.e. these nilpotent ring elements.

yup not at all - i just wandered in off the street and knew accidentally that you were talking about non-standard analysis.

> The rest of your points I think I have already addressed

please show me the source line number in pytorch or tensorflow that defines this number

> we add one called "h" with h² = 0!

you literally started out your miraculous comment with

> This new algebra is called the ring of "dual numbers." The difference is that instead of adding a new element "i" with i² = -1, we add one called "h" with h² = 0!

not some observation about caching derivatives.

so i'll repeat myself for the 3rd time: there are no magical numbers anywhere in pytorch or tensorflow or cafe or any other serious autodiff implementation that abide by the rules you so jubilantly exclaim about.

someone quoted from their intro - they claim this is for grad students.

again, i'm on point and right no matter how much hn plays devil's advocate for things it (hn) has a fetish (overly mathematical treatments of tech).

pytorch absolutely does not use the dual number formulation - there are absolutely no magic epsilons anywhere in pytorch's (or tensorflow's) code base. what you're calling duals are the adjoints where are indeed stored/cached on every node in pytorch graphs.

there's a reason no one uses dual numbers (non-standard analysis) for anything (neither autodiff nor calculus itself): because manipulating infinitesmals like this is fraught formal manipulation (it's algebra...) where as limits are much more rigorous (bounds, inequalities, convergence, etc.). my favorite question to ask the non-standard analysis n00bs is: please tell me under what conditions this is true

(dx/dy)(dy/dz)(dz/dx) = 1

edit:

anyone that thinks i'm wrong and this other guy is right should go and do some reading, eg where this guy tried to make this same point and got shot down:

https://math.stackexchange.com/a/341550

spoiler alert: there's a reason you had to learn epsilon-delta proofs and limits and it's not because your math professors are mean.

this is why i hate this kind of "TIL, gee whiz" math tidbits - they're full of exclamation marks and fancy sounding words ("non-archimedean rings" oooo fancy) but almost always come from a wikipedia level understanding, not actual research.

that's not what this book is. like i said in the first sentence right there in the part right before the all caps calculus - this is book is a compendium of theorem/proofs/etc. very little actual software.

> Doesn’t this apply to all books of any kind?

again, i already covered this: a pedadogically sound textbook will have exercises and structure/themes/etc rather than just 120 "propositions".

go check out any real graduate textbook. what you will find is they all have exercises.

> It's a book for ML researchers. I'm excited to read it. Calm down.

just because the authors claim something doesn't make it true. i'm not wrong - this is not a good pedagogical resource and i would bet a year of my salary (as an ML researcher) that you will in fact not read more than 5% of this book.

actually i know why people write these but i still don't know why they publish them: this is a phase everyone goes through in their "math" life where they look back on everything they've learned hastily between undergrad/phd/postdoc (or whatever) and they have the urge to formalize/crystallize. everyone has the urge - i had a late-career QFT prof tell me that he was excited to take his sabbatical so that he could finally do all of the exercises in peskin&schroeder for real real and type it all up neatly.

i've done it too, in-the-small (some very nice notes that i'm proud of, on various things). you sit down, make your list of things, pull up all of the books/papers you're going to use as references and you start essentially transcribing - but you tell yourself you're putting your own spin on it (adding ample "motivation"). and it's all fine and healthy and gratifying for you and yourself alone. but i don't think i'd ever imagine to myself "well hmm my organization of these topics is going to be useful for other people i should put it out there for the world to see". but that's just me.

ninja edit:

before someone jumps down my throat about "what's the harm?". the harm is n00b/undergrads/young people/etc will look at this and think this is the right way to learn this material and some of them will even make an attempt to learn the material from this thing and they'll struggle and fail and be discouraged - i speak from experience! it's not a good thing for the community. sure maybe 1 in a 100 can learn this stuff from just reading a monograph (what these things used to be called...) but that's the exception that proves the rule.

Then you shouldn't have led with that? It's like academic clickbait to say "here's this thing I'm working on, it uses XYZ technique to solve REALLY HARD PROBLEM" but omit the "...very poorly" part. Like I said I don't entirely blame you, I blame the academics that have an entire cottage industry and culture around these kinds of "results".

> These sorts of Ising architectures get their speed-up are based on oscillator interaction, not parallelism.

Not sure what you're trying to say - it's quantum annealing "brought to life" using simple 2-state (up/down) particles/phonos/whatever you want to call them. The quantum in quantum annealing means superposition ie parallelism; your implementation is basically a D-Wave machine without the qubits.