adrian17
Just for completeness: the original JIT PR was merged quite enthusiastically and only after the fact it was pointed out that it should have went through a PEP process [1]. The following PEP 744 was of the “informational” type, and the just-withdrawn PEP 836 was the first “standards track” PEP, which means that (in my understanding) it’s technically the first actual PEP that the SC would formally comment on. So in a way it’s not “can we update the JIT” PEP, but the first real “does SC want this JIT as proposed” PEP.
That said, I do agree with most of what you said, especially the acceptance criteria part.
The issue I have and find it hard to resolve is that since the code was already on main, the feature was already in “sunk cost” mode. It’s hard when the code is already on main and another 3 optimization PRs are always there behind the corner, but it needs to be possible for SC to say “we decided it’s not actually worth all this complexity for <some small number>% speed improvement” (I’m not commenting whether in this particular case it’d have been a right call), and the PEP (if not withdrawn) was the way to exercise that control.
[1] https://github.com/python/cpython/pull/113465#issuecomment-1...
> The vast majority of JVM optimizations do not get a JEP.
In Python too. Still, the JIT really is just a really big thing. I don’t know if there could be a better way to do it, as either 1. You develop it off-branch and… make an enormous PR when you’re done, with risks of getting rejected anyway? Or 2, you allow more experiments on main like Rust, but then you end up with hundreds of minor half-maintained nightly-only features, possibly conflicting with each other, and eventually go through bureaucracy to remove them anyway.
FWIW, I definitely can find a bunch of both PEPs and JEPs for internal optimizations, random examples:
I don’t really think that’s true or even that relevant , at least in context of classes (especially with slots - the class, by definition, _is_ the “hidden map”) and not generic dicts.
> The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement.
Pretty sure the motivation was exactly the opposite - that a simple “if a LOAD_ATTR encounters the same class with slots N times in a row, swap it out for LOAD_ATTR_SLOT with inline cached slot index, and swap back to generic one if something changes” is way easier (both to implement and reason about) than any alternative that’d interact with the object model. (Also it means you skip having to branch between a slow and fast path at all)
> you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Sorry, that sounds a bit vague and not too different from what CPython already is doing.
Python already does it in a lot of cases, see opcode specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
No clue about details, but Sebastian Raschka's site reports its kv cache usage at 210kB per token, which is like 6x more than this site claims ("0.273 GB per 8k"). Also, the site claims qwen 27b to use 2x more memory per token than gemma, which is the opposite from what I know.
If I recall correctly, a recent post [1] has shown that Q2 quants (with like 2.6 bpw) of the same base Qwen model sit at the edge between "noticeably worse" and Q1's "useless". I took a quick glance at Bonsai's blog posts, and don't really see them comparing themselves to "typical" quants or explaining what's the special sauce that makes them better?
Fun fact, this _used_ to be the case - wasm32-unknown-unknown had extra non-mvp features added later, which in my eyes has been a breaking change on stable. You're not the first person to have been bitten by this, there's [1] and a similar story in Ruffle [2].
Weirdly, the response from the rustc side (IIRC we also talked with one of the devs of wasm component on discord a bit around 2024) has contained a lot of (paraphrasing) "in retrospect we think wasm32-unknown-unknown target was designed badly*, so we'd prefer if you either dealt with it or switched to another wasm target <with its different set of downsides> than us fixing it". (*I acknowledge that the way std was implemented on that target was unusual and "not clean", but it works _really well_ for a lot of people.)
> I looked into the process involved for rebuilding the standard library twice: once with only MVP wasm features enabled and once with an "all yes config" like usual. Based on some research I did this seemed like a massive pain.
That's what Ruffle does [3], it's a bit unfortunate but wouldn't call it a massive pain.
[1] https://internals.rust-lang.org/t/can-we-have-either-a-new-t...
[2] https://github.com/ruffle-rs/ruffle/pull/18397
[3] See lines 55 and 117-127 in https://github.com/ruffle-rs/ruffle/blob/6890c618daab3153343...
> art
Small pushback here: it (and bluesky too) is absolutely atrocious for art, at least as a consumer. Once in a while I stumble upon an artist who only posts their drawings on twitter/bsky, and it’s pretty much impossible to browse their historical works because the „media” tab is just an infinite scroll without any organization; and that’s assuming they don’t also post 10 photos of their cat or other life updates in between their art.
It’s obviously still popular for posting art for many other reasons (I just dislike putting „X”, „art” and „does well” in the same sentence). So IMO the replacements - and lots of these use cases already gave replacements! - struggle simply because of network effects, not their inherent quality.
Might be the first time I see someone complain about their website being protected from a scraper, instead of the other way around.
I saw some eve-specific logic in Destiny repo, like warp enter condition and warp velocity math, or entity visibility between grids.
(Also, it’s full of std::(unordered_)map/set. Surprised they didn’t try squeeze some more perf there.)
The extra diff files are only there because the quantization is not fully reproducible*, and author's 3-line llama.cpp fix PR** supposedly fixes this. With it fixed, then in my understanding the author's tool is literally just a wrapper around llama-quantize.
* I also wonder if this even matters for quant quality. In fact, isn't it possible for contracted FMAs to be more numerically accurate (even if less correct from IEEE pov) than non-contracted math?
** I'd give the PR like 50% chances of not surviving the day, since AFAIK on llama.cpp, vibed PR descriptions usually end up rejected as a rule. Maybe the diff being super trivial could save it from this fate, dunno.
The "dissertation" linked there (https://github.com/b7r6/cassandra-dissertation) is also incredibly interesting; looks like the HN user asked an LLM to prove/validate that they are "right" in their comments more often than not.
In general, these GH accounts and their repos/gists are kind of a rabbit hole.
I buy a VPS. I apt install nginx. Is it okay that by default, opening http://IP/index.html logs the IP address to /etc/log/nginx/access.log? Maybe yes, maybe no, maybe yes but I need a privacy policy (for an empty index.html). Maybe I need to ask a lawyer (who usually errs on side of caution) because people have been arguing about it for 10 years (and please don't answer here). And in the end, even if I didn't need to do anything, it sure is _some_ nonzero drain of my resources to have think about it at all (completely ignoring whether it's justified or not).
There's another issue that popped up on my quick naive profiling run: std::shared_ptr<Material> in the HitRecord/HittableLightSample is assigned/copied and destroyed a lot, and somehow these refcount operations show up as half of all samples on my profile (presumably because even if there's no hit and the pointer stays nullptr, the smart pointer still must check if there's anything to deallocate).
> If the resulting pointer is not correctly aligned for the pointed-to type, the behavior is undefined.
However, unless I’m missing something, producing such a pointer from an integer is apparently not insta-UB? 6.3.2.2.5:
> An integer may be converted to any pointer type. Except as previously specified, the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation
And later on 6.5.3.2.4:
> If an invalid value has been assigned to the pointer, the behavior of the unary * operator is undefined.
Which implies that the invalid pointer must have been obtained without being already undefined, right?
> First off, templates are the opposite of opaque due to the fundamental requirement that the implementation be visible to every translation unit using a template.
That's not strictly true, you can have an implementation hidden in a separate TU, as long as that TU instantiates the template for all template arguments that the users are going to use.
(*hopefully I didn't misunderstand the situation)