oofbey

The thing I find most surprising about this is that the content is in a completely new medium/language compared to what it’s trained on. It’s trained on natural language and code and equations and DNNA sequences. I guess this is just finding the generalization between that and many layers of learned features in high dimensional spaces.

Lots of things bother me about my terminal emulators, and it's almost always in the category of "things don't work right". It's never "I wish my terminal emulator ran faster." Ghostty seems hyper focused on speed, with the negative side effect that compatibility breaks - in practice. I get that it's following all the newest standards or whatever it's doing, but in reality it always needs custom setup to work right. And for me the net effect is that it's always creating problems, not solving them.

You’re right I’m overly dismissive. But there are dozens of papers like this showing some more efficient alternative to attention. There is plenty of evidence pointing to this paper going nowhere.

Efficient alternatives have been coming out for 10 years now. Flash attention is the only one that has really moved the needle - because it’s not an algorithmic compromise. It’s just a smarter way to put the parentheses around the linear algebra operations.

They only demonstrate this idea on a laughably small toy problem. An 18M parameter model (which they call “moderate”) on an ancient dataset proves exactly nothing. Plenty of bad ideas appear to work at trivial scale.

Arguably the most important aspect of transformers is that it allowed easy training of arbitrarily large models.

> a fundamental misunderstanding of the economics at play

I’d say you’re the one who doesn’t understand the economics at play here. To me and I’m guessing the OP it’s quite simple. Have the exploits become common/simple enough to be worth $500 yet?

Why are you hoping for an RCE PoC? Curiosity's sake? Or because you'd like to use it offensively? The existence/publication of such a PoC makes the world less safe.

I’d say this is reasonably severe. For a decently large class of servers exposed to the internet there’s a pretty simple way to RCE them. The actual set of vulnerable servers isn’t nearly as big as many would like to suggest. But if you’re running a node/express server, you should really be in panic mode right now.

I hear you in terms of how these kinds of mistakes typically happen. But why did the script ever produce two sets of keys? The initial root cause was somebody picked the weak key not the strong key. But what was the reason for ever producing a weak key in the first place? They say the script produced two outputs - what on earth for??

The key generator is a FIPS certified HSM that created a strong key. They ran the key through some stupid script that zeroed out all the bytes except the last one. The point of a full FIPS audit is to check that you don’t do anything stupid with crypto like try to implement anything yourself and that you only use certified implementations. But that standard apparently doesn’t apply to CA’s. They only have to use FIPS modules to generate the keys but can do whatever idiocy they want after that. Therein lies the real problem.

Competence is difficult to check systematically. The problem is in the auditing process. The CA auditors check a much shorter list than an application that needs FIPS compliance. They just check that the CA is using a FIPS compliant key generator, and then basic things like following a logged, standard, multi person process.

Anybody writing a system for a bank or the govt needs to have every line of code checked that it’s not doing something stupid with its crypto keys. That level of assurance would have caught this. But the process was defined too weakly.

No. 90% sounds like a low number. That implies something else is writing 10% of the code? People? Other models? What on earth is writing that other 10%?

TVs are getting worse not better. Except for the display. Which should be the only real concern. But the built in software is just going down hill universally.

No. Almost everybody using an iPhone is on the latest release, plus or minus one while updates are rolling out. For a given iOS release and hardware type almost all of those probes are highly correlated. No way there’s 30 bits of entropy in that. I doubt there’s even 10.

Now on a desktop PC sure I believe these fingerprints are highly descriptive. And in android land they probably work okay because of the diversity of products and configurations. But iPhones resist fingerprinting just because they all give the same answers.

It’s a question of entropy. If your device is unusual in detectable ways, then the fingerprint has high entropy. But on my iPhone I suspect most of the things being fingerprinted are the same. How many other people get “smug koala”?

What happened to the CIA world factbook? It was great.

> Information should be free.

Healthcare should be free.

Food and shelter should be free.

“Should” is doing a lot of work there. Meanwhile in reality, creating and organizing information requires scarce labor which deserves compensation.

How about a morsel today? It’s topical.

Two teams have independently gotten a new kind of ultra precise clock working based on isomers of the thorium 229 nucleus. Currently not as accurate as atomic clocks but they have the potential to redefine the metric second.

Absurd

Probably. But also the agents are finding flaws in the security model.

Also Meta is actively encouraging users to grant them full permission, insisting with all their marketing might that it’s safe, which they know is a complete lie. Hard to blame the user when they’re being actively deceived like this. Other agent companies are more reserved and say things like “be careful” but Meta is the opposite.

This is super yawn-worthy. Instead of backprop for the exact gradient you can run forward passes a thousand times with perturbed weights and get a Monte Carlo estimate of the gradient. Not very clever. Extremely NOT useful.

But I guess the industry is littered with techniques for computing the same thing but vastly slower that some people find interesting. Homomorphic encryption. Zero knowledge proofs. Blockchain computing. Except in those cases there might be a legitimate reason to use it occasionally.

It’s not virtue signaling. It’s research. Pushing the limits of hybrids has downstream benefits for consumer cars, presumably.

I agree the current design is pretty dim-witted. But I think the high-level goals are sensible. It’s certainly more than just appealing to the green crowd, because that obviously is misplaced for this audience.

Omg this paradox has bothered me for so long. I’m thrilled to have it finally explained.

Claude also has very arbitrary and confusing rules about what web pages it allows itself to look at, and how much of the page it can read. Did you know for example you’ll get a much deeper analysis if you download a PDF yourself and upload it to Claude instead of giving it the url?

This is a key reason why I actually like Grok for factual queries based on web grounding. It’s fast and reliable. Maybe it’s ignoring robots.txt? Dunno. But it works well.

Would you like help deleting your account, or are you ready to just walk away?

Why did they work in a motel room? The article kinda implies it had something to do with Trump’s funding cuts, but unless I missed something the connection wasn’t made explicit.

In the research literature this is generally called coconut after the first paper published on the idea. It has advantages and disadvantages. I agree it’s more efficient. But the lack of observability and transferability are real downsides that block its adoption.

Right. So why don’t we see residential construction costs fall with this standardization? Because wright’s law / economies of scale don’t work when you are doing all the assembly on site.

You can’t standardize manufacturing of multi billion dollar facilities. Economies of scale just don’t work that way. Even residential housing has the same problem. There’s too much variation in on-site construction.