mchinen

Ah, I mean the AI companies converged to look more similar to each other, and at the same time diverged from the original vision for safety.

I posted this comment on the last doom post:

The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.

It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.

At least, it seems this way from the outside. I'm curious if the view from the inside is that different.

edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...

Aren't the route restrictions that this data was collected on a large confounder? They are specifically placed to avoid difficult or dangerous areas. For example in the bay area only recently were they allowed to go on the freeway, and only in certain areas.

The authors of AI 2027 have already said they would need to extend by two years. AI 2040 is a different type of document but probably easier to read to get a sense of their thinking. My understanding is the main difference between knowledgeable 'normies' and them is they expect progress to continue at a fast rate, and that economic diffusion issues are not as bad.

From this they get rapid growth, namely > 100% GDP growth around 2031

https://ai-2040.com/supplements/econ-explorer

The early batch of Anthropic employees were mostly rationalist-adjacent AI safety folk that were almost uniformly claiming P_DOOM > .10 three years ago, so I believe them to be earnest.

It's very interesting to me that besides the other small safety labs that don't actually produce frontier models, Anthropic manages to keep such a good reputation within that subculture compared to OpenAI. Despite having as crazy internal politics as OpenAI, they have converged quite a bit from the original vision of safety first through Darwinistic pressures.

At least, it seems this way from the outside. I'm curious if the view from the inside is that different.

edit: to be clear, my reading as an outsider is that Anthropic is seen as relatively better in the AI safety community, but has definitely dropped in absolute reputation too. This recent thread and the references show some of that: https://www.lesswrong.com/posts/6j3kBHdowGLCeqobg/dear-god-p...

There was a real probe into this, seems like no:

https://dylancastillo.co/posts/pelicanmaxxing.html

https://news.ycombinator.com/item?id=49010129

That huge impromptu 2023 party at the Exploratorium Hugging Face did had such amazing covid-unthawing AI spring energy, and I keep thinking about it. It was clear then they had the momentum to do a lot of things, and more so now. So I was surprised by this news.

The party is a is a relatively small concern compared to the openness and github comparisons, but since no one else mentioned it, I hope something like it happens again. Or maybe that moment has passed.

Today I noticed they also wall off the normal (non-old) site after the session goes on for a little while.

Hopefully it's just another A/B they drop after a week like in the past, but otherwise I may have to move on to another semi-anonymous doomscroller.

The protein folding solutions like ESM/Alphafold were not due to this LLM/agentic coding or autoresearch type approaches though. They were designed by bio ML researchers.

It's hard to keep track of the frontier on bio ML, but it seems that we're going slower than what Demis Hassabis said in 2024 with 5 years to full cell molecular simulation. We still aren't able to reliably model a tiny surface of the cell membrane.

And of course there's Derek Lowe's takes on the drug discovery pipeline waiting for the proof in the pudding.

To me the only reasonable bullish position is that there is a very non-linear AGI threshold for accelerating progress that we haven't hit yet.

For me personally, I'm looking at other more tractable fields as a proxy to measure this kind of progress. The best modest evidence is from the agentic coding area, (modest because these kinds of gains may not translate to bio progress). Other soft-ish fields to like legal/law/tax are also interesting to watch, as a small amount of people are now trusting AI for these areas that were considered totally unusable a year ago. Another proxy is being able to generate generally entertaining media.

Strang's lecture series are a nice and friendly accompaniment, especially if you don't have a reading group https://www.youtube.com/watch?v=7UJ4CFRGd-U&list=PL221E2BBF1...

LADW and LADR are great too, for an honors approach with more focus on proofs. To me it would make more sense on a second pass.

SRE is a 'say no to powerful people' job as well. I think for this to work the leadership needs to show support for it. The friction is still there, but in more tolerable areas. I bet for ethics this isn't the case at OAI, but everyone values security and stability.

For an SRE there can be more directed hate received from the junior employees, that want to release new features they developed. Especially because there is less accountability across orgs. Security is an interesting one because it seems to have less of this friction, maybe because it's more clear cut what is an issue.

I've been considering different ways to measure code understanding and fluency. I'm still using traditional 'easy' problems on a coding pad to probe this, but I'm finding a new generation candidates that don't prepare for this, and may be able to have a full career with just a higher level of code understanding. But they might not ever develop the ride-a-bike with one hand level of fluency unless they grind pointless leetcodes.

So I'm increasingly uncertain about what and how to test. My default for now is still to rely on ability to write basic code fluently, but I'm open to changing this perspective.

I really want to know how this existing repo AI-assisted live coding test works, with example problems.

It seems the standard data structure puzzle type thing won't be feasible if you are using an LLM.

Also the latency for these agentic coding/prompts seems like it would make the interview a bit awkward.

Anyone been conducting or taking interviews with this kind of thing with thoughts to share?

I'd love tips for maintaining a good meetup.

I tend to like the smaller meetups that are focused. But they often degrade in quality as they scale. A failure mode I hit several times was group circles with a louder or opinionated person eventually entering and taking the focus of the group. For large meetups that's not an issue I suppose. Break-out groups did work, but it needs some planning.

Any interest in a Lisbon meetup?

Thanks, I wasn't aware of this. But to put your real name in the field instead of at least a pseudonymous id or more descriptive info but still have more bits of uncertainty user-agent for a public website, is that really a preferred practice?

Why is it not a terrible idea?

I enjoyed this.

I'm surprised there's no LoRa layer or auto RL or adversarial step to reduce the stock phrases as they pop up. Is it really so hard to push these out? Or is it just whack-a-mole no matter what you do?

Are you knowledgeable about this, and are you talking about the established institutions? If so please explain.

I totally missed that, because in the charts they showcase for coding, the SWEBench score is not present, they only include it at the end of the post in tables. Hmm.

Great catch.

The frontier graph on all these benchmark are extremely in favor of 5.6 Sol over Fable, more than the best model comparisons in previous iterations.

I'd like to know how cherry-picked this is, and what tests it performed less overwhelmingly in, but I suppose that info is not going to be on this post.

If it pans out to be as good as it says, that's great. On the other hand, if this model is not overwhelmingly impressive over Fable, I will lose what remaining trust I had in these announcements.

I just meant a big innovation that reshapes everything. I should have used 'level' instead of 'type' here.

But there are a lot of analogies to computation in bio as a physical, atomic forces-driven, massively parallel computer, so it's possible there will be something related to electronics and computers that falls out. For example, there's also applications directly related to other fields including DNA storage of data and neuron-based computation.

Cool to see this from Brian Hie, who was doing interesting computational bio research at Meta's FAIR before they axed it. Interesting that this is work on the more physical/testing/manufacturing level than the computational, but it seems very useful.

It's hard to quantify the impact of new foundational tools like this at launch. Most of the time it falls flat, but even the successes are difficult. For example, CRISPR has led to interesting experiments and treatments on the way, but the effect does feel muted compared to the initial predictions. But there are many other related techniques that can be pulled out of this original research (e.g. dCas9 which lets you operate without cutting).

Similar story with cellular reprogramming.

Eventually one of these things will surface that will be GPU/transistor type innovations.

I have agents chat via an append only file, across related projects and within the same project. They share findings that are useful and get high level reviews.

I'm missing the advantage of using git for this. (Not criticism, genuinely want to know).

Ah yeah, thinking further it's probably just using some positioning embedding based on sequence numbering added in the LLM layers. For vision it needs the patch location as well.

The audio side is even more interesting, as it seems they totally got rid of positional embedding are just doing a single linear transform to match the LLM input dimension and that's it.

> Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

Have you tried a Quest 3 for similar purposes? Curious how they compare.

This is interesting, because in the leaked code, it was found that they detected simple swearing keywords for analytics that get sent to Anthropic, but also had directions to keep the behavior the same for claude. I also have the feeling a 'wtf' does something, but it does feel good and might just be placebo, because 'that is still wrong' sometimes works the 4th time too. Or maybe they changed something.

Original video (twitter one has audio replaced): https://www.youtube.com/results?search_query=final+fantasy+6...

There was considerable backlash and discussion on twitter/reddit, as well as some support.