magusdei

There's plenty of retrieval-based models that do cite sources. They just didn't want to deal with it for this release.[1] I'm sure it's already on the roadmap.

[1] In fact, some snooping suggests they specifically disabled that feature, but do have it in test environments. See the "browsing disabled" flag they have in the hidden prompt. That could easily be used for citations. Source: (https://twitter.com/goodside/status/1598253337400717313)

No, it would take approximately 3 minutes where you either write a paragraph in the desired style yourself or paste one from the Internet and then ask it to continue in that style. Even if you decided to go the more onerous fine-tuning route, it would require 1-2mb of text, cost very little, and you'd be done in a few hours. It's easy.

"Ego sophos, ergo et qui mihi similiter cogitant sophi sunt"?

>AI anime character generator game startups that YC seems to enjoy investing in

Wait, YC invests in this space? If it isn't any trouble, could you point me to a few?

>If you have details, let me know.

Athanasius is greek for "immortal" and is a common name of saints, church fathers etc. Lubensk (Лубенськ) is an obscure city near Luhansk. Saint Athanasius of Lubensk refers to Athanasius III of Constantinople, saint of the Orthodox church known as an "enlightener", who died near Lubensk. According to the Wikipedia article[1] he sometimes appears to people in dream visions, so your experience is not entirely without precedent.

[1] https://en.wikipedia.org/wiki/Athanasius_III_of_Constantinop...

According to myth, Tamamo-no-Mae would attach herself to emperors and drive them mad, causing them to commit massacres. Hmm...

Your feelings are understandable, but I'd say you will be more effective in actually helping people if you manage to develop compassion even towards people who seem hate-worthy. I wish you continued success in your practice.

Then, would you say your actions are in line with the compassionate awareness of bodhicitta?

"Monks, do not wage wordy warfare, saying: 'You don't understand this Dhamma and discipline, I understand this Dhamma and discipline'; 'How could you understand it? You have fallen into wrong practices: I have the right practice'; 'You have said afterwards what you should have said first, and you have said first what you should have said afterwards'; 'What I say is consistent, what you say isn't'; 'What you have thought out for so long is entirely reversed'; 'Your statement is refuted'; 'You are talking rubbish!'; 'You are in the wrong'; 'Get out of that if you can!'

"Why should you not do this? Such talk, monks, is not related to the goal, it is not fundamental to the holy life, does not conduce to disenchantment, dispassion, cessation, tranquillity, higher knowledge, enlightenment or to Nibbana. When you have discussions, monks, you should discuss Suffering, the Arising of Suffering, its Cessation, and the Path that leads to its Cessation. Why is that? Because such talk is related to the goal... it conduces to disenchantment... to Nibbana. This is the task you must accomplish."

-- Viggahika Sutta, SN 56.9

Sorry that my answer comes so late, but I'll put this here for posterity. I will only address the latter part. My point was that a brain is an N x N -matrix in the same sense as an ANN. An ANN is no more an N x N -matrix "in reality" than a biological brain; in reality it is some collection of analog electric potentials and configurations of matter which can sometimes be represented as a digital N x N -matrix for the convenience of the programmer. Thus the situation is exactly identical to a biological brain, which is also not "in reality" an N x N -matrix but can be represented as one. If we had sufficiently advanced (nano-)technology, we could manipulate human brains through their abstract representation as a matrix just as we can ANNs. Any distinction is purely pragmatic.

In any case I was not saying that being representable as an N x N -matrix is sufficient for consciousness (which I do not believe), simply that it is clearly compatible with consciousness. I agree that a self-aware ANN would probably require a body (possibly simulated) and some notion of agency.

It depends on what you'd consider fundamental. It's true that most of our advances since the mid-80s have been about improving the robustness, data- and compute-efficiency of the training process through optimized architectures and learning algorithms, and in principle in the limit of infinite data and compute you could have taken a model from 1986 and scaled it up to do everything that our current models do. In that sense there have been no "fundamental" advances.

On the other hand, in the limit of infinite size and complexity most mathematical functions can be represented by hash maps, yet to say that there have been no fundamental advances in programming since the invention of hash maps in the fifties would seem like an odd claim to make.

There's a good reason for this -- the version of GPT-3 used for these tests was fine-tuned on CYOA adventures, where this kind of sudden death is very common.

Your commitment to empiricism is admirable. What kind of evidence would you find sufficient?

If you know what a Markov chain is then you must also know that modern language models are nothing like Markov chains. Just as an example, a Markov chain can't do causal reasoning or correctly solve unseen programming puzzles, the way GPT-3 can.

As for self-awareness, your brain is an N x N -matrix in the same sense as an ANN, so surely it must be possible for one to be self-aware? Not claiming that GPT-3 is, of course.

Ending up "net ahead" does not necessarily mean maximizing the fraction of successful projects. I don't think pg is saying that implausible-sounding projects usually turn out good. He is saying that when they turn out good, they have outsized impact, precisely because they initially sounded implausible and therefore produce lots of new information if true.

Wouldn't the empirical success of GPT-3 in simple programming tasks itself be evidence against this interpretation?

Furthermore, GPT-3 is only a language model because it is trained on textual data. Transformer architectures simply map sequences to other sequences. It doesn't particularly matter what those sequences represent. GPT-2 has been used to complete images, for example: https://openai.com/blog/image-gpt/

You can play in a similar way with code. It's just that other developers will hate you for it.

Context-free grammars were originally invented by linguists for analyzing natural language. You can see examples of phrase structure and dependency diagrams e.g. here: https://en.wikipedia.org/wiki/Dependency_grammar

For what it's worth, I agree with OP, and have a very "language-oriented" thinking style. I certainly don't visualize anything while programming or doing math (except for geometry and the like). My thinking feels like it's more based on constraint solving and seeing analogies between domains.

I don't think so. Uncurated GPT-3 output is generally very noticeable because it is unable to hold coherence for an article of this length, and will often make the same points over and over in slight variations and sometimes even contradict itself. I see none of that in this article. I do agree that the general "feel" of the text is similar to reading GPT-3 output, but this seems to be just because the author has a somewhat rambling argumentation style. (I can relate; my own tendency is to ramble on loosely related topics when writing down my thoughts.)

This could of course be curated GPT-3 content, but sufficiently curated GPT-3 output is indistinguishable from human writing, so there's little reason to criticize the article on those grounds.

edit: Well, inspecting the site some more, I agree that there's a moderate chance that this is a test run for a process or a program that uses some kind of novel technique to get GPT-3 or some other LM to produce more coherent essays. I'm still about 80% confident there's a human in the loop, though.

Hmm, this still doesn't quite answer my question at the level of concreteness I was looking for. But thank you for clarifying.

The thing is, you can only define predictive accuracy relative to some experimental design. Otherwise you can always claim that there is some unknown, unperformed experiment where the predictions of the model and your actual behaviour would diverge to a greater degree than is permissible by your accuracy threshold, no matter how many successful experiments have already been done in constrained conditions.

Imagine a task where you have to classify images as being of dogs or non-dogs. We can already train a model that can almost perfectly predict the choices you would make during the runs of such an experiment. But we obviously wouldn't call such a model a "model of your brain"!

My question is this: what would be a sufficient experimental design or empirical criterion to decide that some program is a model of you? The loosest criterion I could imagine would be something like "can successfully deceive your loved ones into believing they are you in a single text chat of unbounded duration with some extremely high success rate." Recent advances in NLP lead me to believe that we'll be able to reach at least this level of fidelity quite soon.

Can you specify what you mean by "emulate my brain within an acceptable margin of error"? What would this mean as an actual experiment? Depending on your answer, I think we can actually test your implicit proposition that no such algorithm exists.

I've actually done this experiment by putting a GPT-3 bot in a Telegram group. Its replies were mostly stuck in an uncanny valley where they sort of made sense, but often either lacked detail or seemed to very slightly misunderstand what the topic of discussion was. This might have been just because I didn't include enough context in the prompt, however. I have some plans for improving the prompting strategy, so we'll see.

I actually recently wrote a post on the topic of whether GPT-3 can be said to understand anything[1]. The argument is a bit too long to summarize here, but I don't think what GPT-3 is doing is as fundamentally different from what human brains do as people seem to think.

[1] https://magusdei.com/why-gpt3-can-understand-things.html

You are of course free to model the underlying causality as you see fit, but the fact remains that this is one of the most robust and well-understood findings in all of psychology (see e.g. the Rescorla-Wagner model). The effect does not depend on the type of reward and exists in practically all intelligent animals, including humans. As noted, video game compulsion loops and gambling machines are carefully tuned to take maximal advantage of this "variable reward ratio".

I like to model this in terms of multi-armed bandits. Given a set of levers giving out unknown rewards, what is the optimal policy to maximize your rewards over time? A bad way would be to try all the levers until one gives you a reward, and then just keep pulling that one lever in hopes of more. This doesn't work because the other levers might have given you even better rewards.

Instead, you should try to learn to predict how much reward each lever is going to give you. A fast way to do this is to focus on pulling levers that "surprise" you, i.e. where your predictions of reward deviate from the actual reward you got. This works as long as the reward from the environment is at least in principle predictable, as it mostly is in nature. But with truly random rewards, you tend to end up with addictive behaviour. In nature this isn't a big problem, because truly random rewards are generally one-off events. So we're basically exploiting a bug in our own reward mechanisms, and evolution hasn't had time to adapt.

Incidentally, all of this can be viewed as a mathematically and psychologically precise way of saying that the reason you get addicted to news is because you are curious.

I suppose. But given the constant inflation in what we demand of AI for it to count as "intelligent", it seems that eventually our tests will become so strict that most humans will not be able to pass them.

I doubt it. Mitsuku, a purely rule-based chatbot, was already able to correctly answer almost all questions of this form in 2014 simply by querying a large knowledge base of common-sense facts.[1] On the neural net side, Google's seq2seq was able to answer questions like this around ~2016-2017, although I have no idea about the accuracy.

It would be more remarkable if GPT-3 couldn't solve these types of questions. It might be another problem with the prompt design.

[1] Incidentally, the article is wrong in claiming that the state of the art before modern neural nets was Eliza. Rule-based chatbots got quite advanced in 2013-2016, although they admittedly were never capable of the sort of "true" understanding and long-term coherence that GPT-3 seems to display.

Yeesh, gwern. Usually I enjoy your comments, but this one seems a bit unfair. Of course if you want to know the long-term future of robotics, you should look to sci-fi authors and research prototypes. But if you want to know the short-term future (<10 years), you should listen to people with actual experience building actual products, even if you don't see those products as impressive from a research perspective.

I think the article provides an illustrative example of the different factors and complications that make it so difficult to go from a research prototype to an actual consumer product (a process which seems to take about 20 years on average). As usual, the lackadaisical improvement of battery technology is one of the central bottlenecks.

Of course, if your main interest is industrial robotics, where DRL approaches are more likely to become mainstream in the near-term, insights related to consumer products might not be as relevant. But note that industrial improvements, while important, will be mostly invisible to consumers except in terms of prices, unemployment and increased customization options for products.

Even then, the article can provide some perspective. Our processes (whether assembly lines or household vacuuming) are already highly optimized for a particular way of doing things, and researchers in particular tend to underestimate the amount of effort it takes to change systems to accommodate a slightly different method. So any new system not only has to be competitive with the existing one in just about every parameter, it has to justify the often huge costs of small alterations to the process. Thus, even in industry, getting to mainstream adoption of RL will likely take quite a while.

Isomorphism != equality. Formulas in greek numerals are trivially isomorphic to ones using arabic numerals, because they are different representations of the "same" concepts, but nobody would argue that arabic numerals aren't more practical (at least for us).[1] So even though the concepts the aliens come up with might be technically isomorphic to (a version of) lambda calculus on some extremely abstract level, there is no guarantee that either we or they would be able to recognize this, owing to a vast difference in representation.

[1] Similarly, a basic theorem in algorithmic information theory states that all programming languages are about equally efficient in terms of program size,[2] but that doesn't mean that all programming languages are actually equally good for every problem in practice, as the constant debates about "the best language" among programmers clearly show.

[2] The basic argument is that, if programs written in language A are vastly shorter than those written in language B, it is possible to write an interpreter for language A in language B of constant length c. So all programs written in language B are at most c chars longer than those written in language A.