jaidhyani
jaidhyani on Hacker News
I used to work for Meta. I quit largely because of intense frustrations with the company. Meta has made a lot of mistakes, overlooked a lot of harms, and made a lot of short-sighted, selfish choices. Many things about the world are worse than they could be because of choices Meta has made.
So that when I say that they really do have a zero tolerance policy for anyone using their internal systems to violate user privacy, it's not because I'm eager to defend them. It's just true (at least, it was when I was there). There are internal systems dedicated to making sure you have access to what you need to do your job, and absolutely nothing else. All content you interact with through internal tools is monitored and logged. If you get caught trying to use whatever access your job gives you for anything other than doing your job, security immediately escorts you out of the building. This is drilled into new hires early and often. For everything Meta gets wrong, they really do take this seriously.
Said company is literally in court against said government at the moment, after said government attempted to designate it too dangerous to do business with.
Approximately no one in the community thinks this. If you can go two days in a rationalist space without hearing about "Chesterton's Fence", I'll be impressed. No one thinks they're 100% rational nor that this is a reasonable aspiration. Traditions are generally regarded as sufficiently important that a not small amount of effort has gone into trying to build new ones. Not only is the case that no one thinks that anyone including themselves is 100% correct, but the community norm is to express credence in probabilities and convert those probabilities into bets when possible. People in the rationalist community constantly, loudly, and proudly disagree with each other, to the point that this can make it difficult to coordinate on anything. And everyone is obsessed with studying and learning, and constantly trying to come up with ways to do this more effectively.
Like, I'm sure there are people who approximately match the description you're giving here. But I've spent a lot of time around flesh-and-blood rationalists and EAs, and they violently diverge from the account you give here.
Compare the trajectory of the US to other industrialized countries.
The best charts I could find on this are from an admittedly-biased think tank, but the sources it's pulling from are well-regarded and neutral:
https://www.americanprogress.org/article/7-reasons-the-u-s-e...
The US also recently passed a massive investment in building out renewable energy.
https://ourworldindata.org/renewable-energyQuick stats for the US:
In 2022, 11.3% of energy was generated by renewables (hydropower, solar, wind, geothermal, bioenergy, wave, and tidal). It's been growing at just under 0.5pp/year since 2007, when it was at 4.4%.
This is primarily driven by wind and solar. Wind power took off around 2000, and in the years since has grown from 5.6TWh to 434.3TWh in 2022. Solar power took off around 2011 and has since grown from 1.82TWh to 205.1TWh. Hydropower remains the #2 renewable in the US, with a noisy-but-nondirectional generation between 200TWh and 350TWh going back to the 60's, but solar appears poised to overtake it by 2024. All other renewables combined are holding steady or slightly dropping at ~75TWh (though anecdotally there may be some large geothermal capacity coming online in the medium-term future that would change this).
Narrowing the focus from all-energy-generation (e.g. including fuel) to specifically electricity, the US is currently generating 22.3% of its electricity from renewables, a number that has been steadily increasing at about 1pp/year since it was 8.4% in 2007.
Naïve extrapolation suggests we're about 75 years out from 100% renewables for electricity, but of course there are reasons to doubt that. For one, we've recently passed the tipping point where renewables are just straightforwardly cheaper than other sources of energy in many circumstances, and improvements in technology and infrastructure will just continue to make this true in more and more cases.
> His actions made perfect sense from his utilitarian Effective Altruist worldview.
They don't. Everyone in EA (AFAICT) has been pretty clear about this. Lying and undermining trust and institutions does tremendous lasting harm.
I am also tired of "people are very concerned about X and think that it's important, so they're basically a cult".
I will never cease to wonder at how so many people can blame so much on people trying to take a rigorous approach to world improvement, up to and including "a narcissistic con-man claimed to do trying to do X, and I can imagine a scenario
where someone could justify doing the shitty things he did to justify X, so therefore everyone trying to do X must also suck and be complicit in fraud and assorted sins".
I am begging people to stop confusing "I was unable to get LLM X to do Y using strategy Z" with "All LLMs are categorically unable to do Y".
As the other commenter said, this is incorrect. The input was a sequence of legal moves (not even "real" moves - most of the training data was synthetically generated with "generate legal moves" as the only constraint).
Deducing board state from this is extremely non-trivial.
Alternatively, the prior on "this is not possible" is very low because RLHF & Friends have targeted metrics that, inadvertently or not, discourage that outcome.
Going to be extremely hard to quantify, ransomware peddlers aren't famous for their meticulous public record-keeping. You could try to sift through all the transactions on the public blockchain and try to classify the ransomware ones, but that's going to be challenging at best I imagine.
Overall I'm skeptical of the claim. It could be true, or partially true, but demonstrating that would probably require some work. Alternatively, someone could demonstrate that liquidity came from speculators or some other source, which seems potentially less doable but still not easy.
Could have gone with "More Comprehensive Metrics Are All You Need"
GPT-LikeSubscribeAndRingThatBell
This is true in general but not in the use case they presented. If they had explained why a normalized distribution is useful it would have made sense - but they just describe this as pick-the-top-answer next-word predictor, which makes the softmax superfluous.
Prediction happens at the very end (sometimes functionally earlier, but not always) - most of what happens in the model can be thought of as collecting information in vectors-derived-from-token-embeddings, performing operations on those vectors, and then repeating this process a bunch of times until
at some point it results in a meaningful token prediction.
It's pedagogically unfortunate that the residual stream is in the same space as the token embeddings, because it obscures how the residual stream is used as a kind of general compressed-information conduit through the model that attention heads read and write different information to to enable the eventual prediction task.
It depends on the values of the vectors. (4, 4) + (3, 3) results in a new vector (7, 7) which is further away from both contributing vectors than either one was to each other originally. Additionally, negative coefficients are a thing.
The original paper is very good but I would argue it's not well optimized for pedagogy. Among other things, it's targeting a very specific application (translation) and in doing so adopts a more complicated architecture than most cutting-edge modes actually use (encoder-decoder instead of just one or the other). The writers of the paper probably didn't realize they were writing a foundational document at the time. It's good for understanding how certain conventions developed and important historically - but as someone who did read it as an intro to transformers, in retrospect I would have gone with other resources (e.g. "The Illustrated Transformer").
I endorse all of this and will further endorse (probably as a follow-up once one has a basic grasp) "A Mathematical Framework for Transformer Circuits" which builds a lot of really useful ideas for understanding how and why transformers work and how to start getting a grasp on treating them as something other than magical black boxes.
https://transformer-circuits.pub/2021/framework/index.html
That's true, but they didn't go into any other applications in this explainer and were presenting it strictly as a next-word-predictor. If they are going to include final softmax, they should explain why it's useful. It would be improved by being simpler or more comprehensive, but complexity without reason is bad pedagogy.
Thanks, that's a really useful intuition!
TIL. Man, I'm behind on my paper reading.
I am the guy asked and I endorse this guy's endorsements.
The way the article presents this is misleading. The attention mechanism builds a new vector as a linear combination of other vectors, but after the first layer these have also all been altered by passing through a transformer layer so it makes less sense to talk about "other tokens" in most cases (it becomes increasingly inaccurate the deeper into the model you go). It's also not really moving closer so much as adding, and what it's adding isn't the embedding-derived-vector but a transform of the embedding-derived-vector after it's been projected into a lower-dimensional-space for that attention head.
It would be more accurate to say that it's integrating information stored in other vectors-derived-from-token-embeddings-at-some-point (which can also entail erasing information)
Skimming it, there are a few things about this explanation that rub me just slightly the wrong way.
1. Calling the input token sequence a "command". It probably only makes sense to think of this as a "command" on a model that's been fine-tuned to treat it as such.
2. Skipping over BPE as part of tokenization - but almost every transformer explainer does this, I guess.
3. Describing transformers as using a "word embedding". I'm actually not aware of any transformers that use actual word embeddings, except the ones that incidentally fall out of other tokenization approaches sometimes.
4. Describing positional embeddings as multiplicative. They are generally (and very counterintuitively to me, but nevertheless) additive with token embeddings.
5. "what attention does is it moves the words in a sentence (or piece of text) closer in the word embedding" No, that's just incorrect.
6. You don't actually need a softmax layer at the end, since here they're just picking the top token and they can just do that pre-softmax since it won't change. It's also weird how they talked about this here when the most prominent use of softmax in transformers is actually in the attention component.
7. Really shortchanges the feedforward component. It may be simple, but it's really important to making the whole thing work.
8. Nothing about the residual
I think we're on the verge of few-shot believable voice impersonation. Between that, realtime deepfake videos, and AIs being more than good enough to solve CAPTCHAs, it seems like we're at most a few years from having no means of verifying a human on the other end of any given digital communication unless someone figures out and implements a new solution quickly.
When a dangerous exploit is discovered, best practice is to give good actors playing defense a heads up and time to prepare before publicly revealing the exploit, and preferably only revealing the exploit once critical systems have been hardened against it.
Cheap general AI is a potential exploit of everything everywhere all the time. No one is prepared for it.
A billion eyes are powerful. Giving every actor an eye-making machine is uncharted territory. And opening it up is a one-way gate - you can always delay opening it up another day, but you can't un-open it once you've let it out. So you should be very, very certain you're prepared to commit yourself, everyone you know, and the entire human race to living in that world before you make it so.
The content moderators were fired some time ago.
The Ghostwriter expansion pack was next fucking level. Clues hidden in documents to solve a mystery - one of my favorite early childhood software experiences.