landfish

Palisade Research | ML Research Engineer | ONSITE | Berkeley, CA | VISA

We research dangerous AI capabilities to better understand misuse risks of current systems, and how advances in hacking, deception, and persuasion will affect the risk of catastrophic AI outcomes.

See the full job ad and application here: https://palisaderesearch.org/work

We plan to study emerging dangerous capabilities in both open source and API-gated models across automated hacking, spear phishing and deception, and scalable disinformation.

We are looking for people who excel at: - Working with language models, which includes supervised fine-tuning, using reward models/functions (RLHF/RLAIF), building scaffolding (e.g. in the style of AutoGPT), and prompt engineering / jailbreaking. - Software engineering. Alongside working with LMs, much of the work you do will benefit from a strong foundation in software engineering—such as when designing APIs, working with training data, or doing front-end development. - Technical communication. By writing papers, blog posts, and internal documents; and by speaking with the team and external collaborators about your research.

Salary: We’re offering between 150,000 - 250,000 USD, depending on your skill and experience, plus healthcare.

Note: I wrote this post and cross-posted it to the EA Forum, and there are some good comments there discussing the longer-term risks of nuclear war that I didn't include in the original post: https://forum.effectivealtruism.org/posts/mxKwP2PFtg8ABwzug/...

PDF version is free online too. https://www.oism.org/nwss/ :)

The citation for "The Robock group’s models are probably overestimating the risk [of nuclear winter]." are the Reisner et al. articles I linked. My conclusion is just my opinion after reading the back and forth in the literature, but I encourage other people to go through it and come to their own conclusions!

I've heard a lot of people say this. Wikipedia has a decent summary of the history of this position: https://en.wikipedia.org/wiki/Nuclear_holocaust#Origins_and_...

I wrote this post, and the point is to improve discussion of societal risks. As others have pointed out, nuclear war doesn't have to kill everyone to be extremely bad and worth avoiding. I spend a lot of my time thinking about risks from nuclear and biological weapons, and downplaying these risks is not my purpose.

One problem with assuming nuclear war kills everyone is that this discourages anyone from preparing for potential nuclear wars. While of course we should try to prevent it, we should also try to mitigate the consequences in the even a war does occur!

I've written more about risks from nuclear war here: https://www.lesswrong.com/posts/rn2duwRP2pqvLqGCE/does-the-u... and here: https://jeffreyladish.com/one-hundred-opinions-on-nuclear-wa...