It works in GPT-4 (but not GPT-3.5) even without a special tokenizer: Some problems simply solve themselves with increasing scale. But Bard uses a smaller LaMDA model to reduce inference cost. Quote [1] from Sundar Pichai in February:

> We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allowing for more feedback.

[1] https://blog.google/technology/ai/bard-google-ai-search-upda...

I don't think it works reliably over 100 invokations of the prompt. The paper tries to get provable situations where 0 ignored constraints happen/are possible.

I'm impressed that it is working more often than not, but we need provable guarantees.

If you ask 100 humans the same thing, you'll get some mistakes too. I don't think the point is whether it makes mistakes. The Chat GPT response shown above is clearly more interesting and impressive / displays something more seemingly like intelligence than the Bard response provided. At some point scaling these things up enough becomes indistinguishable from intelligence, and at that point maybe it is.

> If you ask 100 humans the same thing, you'll get some mistakes too

Yes, still, it's pretty easy to find a somewhat good judgement human who never does that mistake. If you tell him/her: "This is of utmost importance: ..." — But as of now, these text gen machine learning programs don't totally understand that?