gslaller
gslaller on Hacker News
Posted by
gslaller on Tuesday September 29, 2026 @10:46AM
from the show-hn dept.
https://gslaller.github.io/moe_analysis.html
Did some work on visualising the firing pattern of MoE (Mixture of Experts) modules for two Qwen models. There are some other interesting statistics, such as router entropy, expert reuse from the previous token (temporal consistency), etc., on 80 prompts from MT-Bench.You will see temporal consistency, which differs depending on the layer.
The gap between the 8th module (the last one picked)...
Did some work on visualising the firing pattern of MoE (Mixture of Experts) modules for Qwen models and other statistics. You'll see temporal consistency in the router, as one would expect.
Posted by
gslaller on Monday September 28, 2026 @07:21PM
from the gslaller.github.io dept.
A noob speaking here. Why aren't there efforts to have a memory bank like structure where you attend to a sub set of codes depending on the key(at the attention level)? is this already done with the global attention mechanism (what is it even)?
Having superficial knowledge about data storage/DBMS, perhaps I am just spewing nonsense. But I always imagine that a compaction layer for the long-tail/old data should be a standard, where less accessed(sometimes once a month) is pushed to a s3(but still queryable) like storage.
Thanks for your contribution and maintaining the pion library(I am a user). The thing is that WebRTC usually isn't a complete solution, many don't want to have p2p but a p2s2p(which usually means distributed computing/SFU) solution, which makes developing an application way too complex.
> func Index[S ~[]E, E comparable](s S, v E) int {
After seeing this signature, I think that Go is giving up on it's simpleness principle.
Still a good idea overall, perhaps they did a silly execution and had made efficiencies error which a team wouldn't do now a day(listed some in these comments). They somehow went for growth-at-all-cost instead of increasing their density to check the unit economics of a single location and then move on with the gained knowledge.
Disney did a better job of demonstrating the Vision Pro's ability than apple. This is obviously going to be a hit, and by gen 3 - 4(with a ton a immersive apps) it might by trailing behind iPhone.
Is the value of accreditation that high, making it an USP? i.e. should an edtech go after the universities and school as a source of certification or more after the engagement of their user on their platform? I prefer the latter.
The content is certainly more interactive, but the interaction among users is still not given and the production also requires more resources to produce(I presume). Can hardly see that this kind of service will become the future of edtech.
Posted by
gslaller on Saturday February 04, 2023 @06:50AM
from the self-post dept.
Arguably, YouTube (YT) is the best education technology platform on the internet, yet it is just a non-creative successor of the VHS/TV and not even CD(which had the menu, breaking the linearity of the frames). The bits are transmitted over switches and routers instead of a cartridge with a magnetic tape, improving the latency for the end-user and YT also has the recommendation system which is...