hashmap

What I have been reading makes me want to both agree and disagree with your take here. I think the fact that some of these things are not only established but are much simpler than one would think at first glance yields the much cooler realization that you can not only do a lot more than abliteration, but that humans can do it by hand if the thing is made legible.

From my experience with models when it comes to discernment about particular things, the models, even the best ones, still are not very good. The thing that we are most missing I think is acceleration in creating editing tools for humans that make the otherwise inscrutable both visually legible and interactive.

Thats not what the issue is, it's that they aren't free to job-hop in the way that other people are. H1B keeps their wages artificially depressed and they have a sword of damocles over their heads threatening deportation.

It's an incredibly obvious optimization on the autoregressive model, and converting a training pipeline to adopt it, with how good autoregressive models are now, is as easy as saying "do this but jepa" to codex or claude. It's gonna be like how everything is becoming rust now.

Oops, HybridLink as a paper came out little bit ago, and it was an architecture I more or less converged on in my own work. JEPA is the obvious successor to autoregression once you look at it a little bit - to the extent that you can take a non-jepa transformer and just slap a couple things on it to make it a latent reasoner without crossing the token boundary. It's faster and more accurate, as you would expect from not having to project down to tokens and have the model reconstruct its own latents.

Infantilizing protest is a wormy pastime.

Man, the "protests don't work" energy is way stronger here than I would have expected! Whatever happened to the spirit of disruption or going a different way?

No, you're using the word in the colloquial pejorative sense. This is a protest, and the fact that they are trying to make it visible in the way that they are is what separates it from something like a generic competitor that does something similar.

"Just" performative? The performance is the point.

> It upsets me somewhat that they’re protesting things like lock-in only after being laid off <

This is the nature of people. It is a good thing to realize the leopards are bad, even if it takes getting your own face eaten to notice. Look at the positive a little!

kind of boggles me that the big ai labs arent running zero-shot classification on all their outputs with "does this look like any of these bad things we know people get up to"

"Look, it wouldn't have even mattered if we had done advanced cyber security like turn off internet or look at logs when huggingface was attacked!! See look it can still.. *prompts a shitload of agents, waits 6 hours* look wowwww they still learned how to talk to each other even if we turned internet off!"

"They wouldn't have been able to attack huggingface though, thats still criminal negligence."

"No see this actually means airgapping doesn't even matter theres nothing we can doooooo!"

"...ok, turn them off and delete them I guess"

"Well no of course not everyone else is doing it so like. No courts please just like be in awe"

Not even close, this is a desperate attempt to cobble together a justification for why not taking the most basic security measures or reading logs during the huggingface incident wasnt criminal negligence.

for putting together something like a tool-call decision model, the conditioning will take less time than getting the data for it, which also will not take much time if you keep your session data. whatever fails is your negative contrast, whatever passes is your positive data. the space of tool calls your agents will use in your particular environment is not actually that big, which makes this one of the easiest wins you can pull off.

there is a lot more of me getting to choose what it is i end up doing each day. i find fun in doing those things now pretty regularly. cracking a tough problem can feel as good as jumping out of a plane did. that is to say, my brain is still not a great judge of how relatively cool something is, because even something like scuba diving or fast sports were wildly cool the first time and then the second time it was like "this again?".

i feel much safer though knowing that now when i sometimes get a little bit of the itch, i'll think about it for a few seconds and go "eh, maybe later," and return to trying to make a thing, or play some music, or why is that stupid thing still broken oh its because i broke it.

That is the opposite of dumb, you can use the next token likelihood to macgyver together a jev in a pinch exactly that way, you just would also want to take some steps to scramble up the choices to smooth over bias and tokenization artifacts. llms have favorite letters for example. Qwen likes A and D.

I saw those moving dots and snap-closed the page.

How did you manage describe a litany of automation while being fully unaware that is what you were doing? I mean you even said the word autopilot lmao. Yes, not all tasks in a job are fully automatable. iPhones, automating communication. Quickbooks and turbotax, automating business workflow steps.

The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.

As a software engineer? The career whose very reason, largely, for existing in this market is to automate jobs, with few notable exceptions? That one is somehow deserving of sympathy and that of mathematicians is not? Come off it. The notion is undignified.

Put someone in charge who made their way by underpaying people to sit and label data vs the inventor of convolutional neural networks. Its no coincidence that shortly after his tenure ended, a model that took a long time to prepare started becoming actually useful.

Comments like this one reinforce the obvious truth that the people who are most "pro ai" are the ones who understand it the least

rlhf will cause models during training to hide data in regions not typically accessible to them in the geometry during inference, due to competing objectives, so i wonder if this looks like that.

So far, he is right. No frontier model right now performs well on making intuitive physics decisions when working on novel VR physics interactions. They literally all fall on their faces and it is very frustrating.

You're missing the point here. He's not talking about whether or not they can learn facts or inferences derived from the text itself, but the more holistic intuition that results from learning from something like an embodied experience in the physical world. GPT-6 Astras web demo homepage thing is an example. It chose euclidean rather than quaternion for letting a user rotate the galaxy thing, and anyone who has ever used hands to rotate something would immediately recognize on trying it that something is fucked and you shouldnt do that. Thats the kind of common sense physics that is inherently beyond these llms and I run into it ALL the time in vr programming.

how long until cutting out the token middleman and just keep the cache constant sized but edit it and just send the model new information instead of the same thing over and over. token bad latent good.

Are you arguing that training on copyrighted material is fine but distilling against tos is somehow bad? Lmao what a take. Nah, if copyright cant even be enforced then theres no way on earth violating a tos should ever have repercussions other than they ban you. stop carrying water theyre not gonna pay you

i am a small fan of pg, nevertheless i find this to be an exceptionally good take and it is strange to me to see so much piling on to this one in particular here.

learning about llms is not useful so that you can make llms later, you want to learn about it so that you can work on next generation architectures. llms before long i imagine will be left in the dust by ebm / physics oriented models especially that can have an embodied understanding of the world. but a lot of things you learn about them are transferable by doing something like this

If taste that's all that's left you've kinda really got nothing. Taste is an anti-moat, it's sort of a measure of how well you, as a literal receptor, agree with a pile of people out there on something. Like sure its something you can sort of develop but it costs you nothing to use, and if you're not creating something off-manifold you're just smearing stuff around that's already on it. Horrendous kind of take on "taste" that's taken hold of everyone. Kinda like if a music artist stopped making music to make spotify playlists. Sure they could maybe make _good_ playlists, but if they are sitting there crowing "taste is all thats left!" it's.. what is that? It just sounds like either a deluded giving up or just a very direct admission of giving up.

i would be more likely to click around this page if that specific frog was not involved

At the risk of you hating this answer, because I hated this answer, but to me that sounds like you are holding it wrong. I have adhd and found it dumb and boring and possibly not even real / placebo or new agey until during one attempt I hit the phase change and had the holy shit this is amazing moment where it worked for like 3 seconds. The quiet relaxation felt like idk a euphoria in every part of my body just to not be anxiously mentally bouncing around for half a second.

> every PC component I've owned in the last 20 years. I've attached as much of it to my homelab as is practical

this sounds like a disease. one of the early signs for me was developing a particular interest in the breaker box.

the foamy hedging makes me ill and hurts my eyes