mbowcut2

I thought girls didn't use junior, III, etc. because traditionally girls adopt their husband's last name, so they would no longer have the same full name as their mother.

This shouldn’t be surprising. Weights are in the training set and are necessarily structured. We should be asking the model to extrapolate on the structure.

I feel like you're treating the stock market like a horse race. Amazon outpaced the stock market by 3x over the last 30 years, and it's just incredibly unrealistic to make that the standard for buying a stock. Anthropic shouldn't be expected to increase any more or less than the rest of the market and that's if you don't believe there's a bubble. (A bubble is by definition when prices don't reflect actual value)

So, if you crash and the air tank ruptures do you just... die?

I think the disagreement here is in time scale. The human-in-the-loop period for software development may be orders of magnitude shorter than infrastructure engineering and other physical applications.

So, it's a Helldivers reskin that's going to be 10 years late to the party?

I made myself reason it out, and came up with the exact same intuition. You need a sequence of 2n moves (n down moves, n over moves), but the sequence is completely determined by which moves are down (the others must be over). So it's 2n (moves) choose n (over).

Gotta hit that docker system prune -a

Loved him in Secondhand Lions.

If you thought we were getting bad bugs before, just wait until the 90% agent-coded PRs start landing. We're gonna have multiple crowdstrike-level blowups.

It's an interesting concept, but I'm skeptical about how feasible this is. How much design/legwork/intervention will Seth actually contribute during the entire process? I'm thinking "growing corn" might be a little hard for a proof of concept, specifically because the time horizon is quite long. Something a little more short term like: contracting a landscaping job. The model comes up with design ideas, contacts landscapers, gets bids, accepts a bid. Seth could tell the model that he's it's agent, available to sign for things, walk people through the property, etc, but will make no decisions, and is only reachable by email or text.

Wow, I didn't know about the "skills" feature, but with that as context isn't this attack strategy obvious? Running an unverified skill in Cowork is akin to running unverified code on your machine. The next super-genius attack vector will be something like: Claude Cowork deletes sytem32 when you give it root access and run the skill "brick_my_machine" /s.

It makes me wonder about the gaps in evaluating LLMs by benchmarks. There almost certainly is overfitting happening which could degrade other use cases. "In practice" evaluation is what inspired the Chatbot Arena right? But then people realized that Chatbot arena over-prioritizes formatting, and maybe sycophancy(?). Makes you wonder what the best evaluation would be. We probably need lots more task-specific models. That's seemed to be fruitful for improved coding.

Seems like the less sexy headline is just something about the sample size needed for LLM fact encoding That's honestly a more interesting angle to me: How many instances of data X needs to be in the training data for the LLM to properly encode it? Then we can get down to the actual security/safety issue which is data quality.

I'm not surprised. People really thought the models just kept getting better and better?

it looks like the 2nd and 3rd bar never got updated from the dummy data placeholders lol.

It's not a new problem (for individuals), though perhaps at an unprecedented scale (so, maybe a new problem for civilization). I'm sure there were black smiths that felt they had lost their meaning when they were replaced by industrial manufacturing.

I've had similar experiences with vanilla ChatGPT as a DM but I bet with clever prompt engineering and context window management you could solve or at least dramatically improve the experience. For example, you could have the model execute a planning step before your session in which it generates a plot outline, character list, story tree, etc. which could then be used for reference during the game session.

One problem that would probably still linger is model agreeableness, i.e. despite preparation, models have a tendency to say yes to whatever you ask for, and everybody knows a good DM needs to know when to say no.

You can, and there has been some interesting work done with it. The technique is called LogitLens, and basically you pass intermediate embeddings through the LMHead to get logits corresponding to tokens. In this paper they use it to investigate whether LLMs have a language bias, i.e. does GPT "think" in English? https://arxiv.org/pdf/2408.10811

One problem with this technique is that the model wasn't trained with intermediate layers being mapped to logits in the first place, so it's not clear why the LMHead should be able to map them to anything sensible. But alas, like everything in DL research, they threw science at the wall and a bit stuck.

The problem with embeddings is that they're basically inscrutable to anything but the model itself. It's true that they must encode the semantic meaning of the input sequence, but the learning process compresses it to the point that only the model's learned decoder head knows what to do with it. Anthropic's developed interpretable internal features for Sonnet 3 [1], but from what I understand that requires somewhat expensive parallel training of a network whose sole purpose is attempt to disentangle LLM hidden layer activations.

[1] https://transformer-circuits.pub/2024/scaling-monosemanticit...

LLMs are better at LaTeX than humans. ChatGPT often writes LaTeX responses.

I think I agree with you. My only rebuttal would be it's this kind of thinking that's kept any leading players form trying other architectures in the first place. As far as I know, SOTA for SSM's just doesn't suggest significant enough potential upsides warrant significant R&D. Not compared to the tried and true established LLM methods. The decision might be something like: "Pay X to train a competitive LLM" vs "Pay 2X to MAYBE train a competitive SSM".

I read this as "pirate space industry" and got real excited.

It's interesting how I couldn't tell whether the rocket was 1m tall or 10m tall in this video. Turns out it's actually 6m tall per the link.

Nah, I think they made it model agnostic, which is kinda smart.

To a topologist, everything is topology.

Pack it up boys, they finally made the killer app.

I had a smart TV that gradually got slower and slower until it became basically useless. I figured it was just running out of RAM as apps got larger with updates over the years.