I was really blown away by the results you achieved. Amazing work! My jaw hit the floor when I saw the witty farewell "fun guy" quip, and I was in stitches when I read the song about baking. I look forward to the day I can take the model for a spin - unfortunately I don't have the requisite $18,000 hardware ;)

I have a few questions: Could this be used as a tool to get a feel for public sentiment? For example, could you ask the bot what it thinks about gun control and have it spit out a policy that appeals to the common public? If you ask the bot what it thinks about how a company will perform, how accurately does it predict? I know that the model will contain the biases of the data set, but I'm curious if you've run these types of experiments. What do you think the results would be if you had an even bigger, more diverse corpus? (devil's advocate, for the sake of discussion: perhaps everyone's fb messenger and WhatsApp chat history)

Finally, you have clearly gone to great lengths to make the bot pleasant to interact with. What sort of results to you get when you train such a huge model on an uncurated corpus and don't try to tweak its personality? I find myself wishing that you didn't try to do this as the bot seems to be hyper-agreeable. I. E Too many responses like "You like watching paint dry? That's super interesting! I love watching paint dry!".

I would not encourage using the model for anything other than AI research -- we're still in the early days of dialogue, and there are a lot of unexplored avenues. There are still nuances around safety, controlling generation, consistency, and knowledge involvement. For instance, the bot cannot remember what you said even a few turns ago, due to limitations in memory size.

In the paper, we did explore what happens when you do NOT fine tune it on the specialized tasks (knowledge, empathy and personality). The non-finetuned bot was both less engaging and more toxic. The special finetuning is really important to getting this bot to be as high quality as it is.

But toxicity and quality is subjective. The technical achievement is undeniably brilliant, but the quality of the personality is subject to opinion - as I mentioned, I did not personally enjoy the agreeability of the bot. What's toxic today may not be toxic tomorrow and vice versa.

It's just a matter of time before a model of this size can be run on commodity hardware and somebody will take the brakes off and/or attempt to run experiments that aren't just "can this thing pass the turing test?". I'd be really interested to know the thoughts of the team, given their expert knowledge and experience with the matter.

For a more toxic version of a similar kind of bot, check out SubSimulatorGPT2: https://www.reddit.com/r/SubSimulatorGPT2/top/?sort=top&t=al...

Unfortunately you can't talk to it. (I've wanted to retrain a version that you can interact with dynamically, someday.)

The comment thread on the self-awareness post is both very convincing and really meta. I love how clear the training of the different bots is.

Was the bot nonsensical without the fine tuning, or just subjectively a worse conversational partner?

I think the GPU price you’re quoting is about 10k too high. $8500 for the V100/32gb

> The 9.4B parameter model requires at least two 32gb V100 GPUs to interact with

plus motherboard, cpu, ram etc.

> Ask us anything, the Facebook team behind it is happy to answer questions here.

Why do you think that Facebook paid for this research?

It's not obvious? Facebook has enough of your conversations to fingerprint your writing style. Now they can de-anonimize your reddit account and serve you even more personalized ads.

That sounds like total overkill. To deanonymize a reddit account, it is easier to just use a add/cookie/script on reddit to link it to your facebook account you have open in another tab.

How exactly would you do that? Reddit doesn't allow random scripts by 3rd parties.

Ok, I don't know about any reddit details, but I just opened reddit.com without a adblocker and took a look in the network tab in the dev tools.

I mean, I don't see a plain facebook connection, but I see googleads and amazon connection and don't know what else obscure things. I doubt it would be hard to sneak something in, that just checks whether this is the same browser where you just switched over from the facebook tab (where the onblur event just got captured). But again, I am not an expert in tracking ads, nor reddit, I just know the web and its various data transmitting technices quite good.

FB has an unlimited budget, a financial incentive, and reddit basically handed them the data. This is not even close to overkill.

This could be rolled out as a service for companies to integrate into their pages to provide support to their users.

How do you feel about this probably being most used by bad actors trying to screw with society and elections?

That's definitely a fair concern. We believe that open science and transparency are the right approach here. By releasing it, we ensure that everyone is on the same page with respect to abilities and defense.

> By releasing it, we ensure that everyone is on the same page with respect to abilities and defense.

This seems like wishful thinking. Having knowledge and having the resources to do something with it are two very different things.

Defending against such an "attack" is much easier if the technology is widely available and many people can play around with it and explore the limits.

So is crafting such an attack. Given that the attacks are obvious, and I haven't seen much word on defenses, the result seems inevitable.

At some point you have to let the rest of society in on it. As technology advances, if you keep the power away from the public, eventually you will split into two civilizations a vast divide creating an asymmetry that will be exploited by those on the elite side of the divide. It must be an aspect of technological rollout to figure out how to keep its usage safe once it is widespread.

Performing the attack is only a possibility because the tech was made available.

You are assuming that such tech is not already being (or has been) developed covertly by malicious actors. Developing this and making it open source brings more awareness about the subject and will make it easier to develop defense models against such bots (whether already in existence or that will be developed in future).

> You are assuming that such tech is not already being (or has been) developed covertly by malicious actors.

I fail to see where playing out the "but others are doing it too" card exempts the responsibility of those who either lower or eliminate the barrier to entry to these attacks.

So what's the defense?

that's the rub.

every AI sound/word/picture editor i've ran into says something along the lines of "we're releasing this data set to help stay secure in this day and age of easy counterfeiting of X.", but they never really mention how you apply the data in an adversarial way against itself -- they just sort of hand-wave that part.

Same with fake AI generated Obama video and sound, and earlier data-set generated chatbots; it's plastered all over the projects things like "Since these methods are available we think that it's important that this data is disseminated so that other's can use it to validate real world data sources", but again -- how?

We have the real data, we have the fake data -- how is this diff done, exactly?

I'm willing to bet it isn't as easy as all the AI researchers who release this stuff claim it may be.

If its secret or not publicly available people will argue using Occam’s razor or that only “State actors” could use this. With the subtext being your not important enough.

With the data public its more akin to driveby ssh login attempts. Not being important doesn’t mean your not under attack and people can take the necessary precautions.

That's a bit like saying that nuclear secrets should be made public so that people can "take precautions" because "anyone can have a nuclear weapon, not just state actors".

There are few reasonable ways to "take precautions" against nuclear weapons and there are few reasonable ways to "take precautions" against something like this short of swearing off of social media entirely.

Without reasonable defences, all you really accomplish is ramping up proliferation.

I don’t think weapons of mass destruction is comparable. More like a security vulnerability for the mind. You can no longer be sure its a human on the other side.

If you're curious to learn more about what's actually being done: https://arxiv.org/abs/1905.12616

If it comes from Facebook, assign it zero credibility?

Everyone =/= everyone unfortunately.

I agree with your approach though.

Sounds like the arguments made by pro gun people.

It's already happening across social media, just with software a whole lot more advanced than this.

On the other hand, what if this technology was used to influence society and elections in a good way?

One of the problem is, who gets to define what “good” is?

Do you believe anyone is in a position to dictate the outcome of an election without the people having a say on the matter?

> Ask us anything, the Facebook team behind it is happy to answer questions here.

Just use the bot. Put it into action. Let the bot answer HN's questions

How long until you have to shutdown because it starts getting offensive like Tay?

The team put a lot of effort on dialog safety. You can find more info here: https://parl.ai/projects/dialogue_safety/ and the paper here: https://arxiv.org/abs/1908.06083.

Still, unlike Tay, we purposely did not create a service for it and do not advise creating one. This is for research purpose only and more effort needs to be made on safety before it can me more broadly consumed.

How consistent is the bot? Will it answer the same question phrased differently with the same opinion/fact/feeling?

One of the tasks that we fine-tuned the model on is ConvAI2 (or "Persona-chat") which specifically aims to improve the model's consistency by conditioning its responses on a given persona. See here: https://arxiv.org/abs/1801.07243. In this research we found that conditioning on a persona improves consistency, but that the models still aren't always perfectly consistent. In particular, the model can only see a few turns back in the dialogue history, and may contradict itself if the conversation is long enough that the model simply does not see the full conversation.

As far as facts go, we also fine-tuned the model on the Wizard of Wikipedia task (https://arxiv.org/abs/1811.01241), which helps improve its knowledgeability. Again, it still isn't perfect.

Would FB possibly put some instances up for people to chat with?

It's way too heavy/expensive to host as an individual.

We are evaluating the possibility. But this is really for research only. To go beyond that we need further improvements to the safety layer.

What does safety mean in this context? (I'm not an AI researcher.)

Maybe that the bot does not start denying the holocaust, like the one from microsoft.

https://en.m.wikipedia.org/wiki/Tay_(bot)

(but I am not sure if this bot does indeed learn from new conversations)

For most the evaluations you reported at engagingness (expect Figure 16). Did you also look humanness? I would be especially interested how human your chatbot is compared to real humans (Figure 17). This would be similar to a turning test.

The bot is really optimized on engagingness rather than humanness - in particular this is how we chose the hyper parameters. We did evaluate Meena vs. Human (31 vs. 69) and Meena vs. BlenderBot (35 vs. 65). But didn't do BlenderBot vs. Human. Good suggestion though.

Does it respond appropriately when presented with a potential "switcharoo"?

It does work pretty well with changes of topic (often integrating the broader context as long as it fits into a few turns).

Anything?

Why do you choose to use your incredible talents for the benefit of such a disgusting company?

I have answered similar questions on HN before. See https://news.ycombinator.com/item?id=19036507 for example.