News for hackers. Stuff that matters.
Re:What Will Transformers Transform? by yorwba on Thursday March 23, 2023 @09:31AM (#35274496)
The answering style matches the usual presentation of a Winograd schema, which is a bit of a problem when you actually want to test its general reasoning ability, because it could be that someone already posted your go-to test question and the answer on the internet and the model was trained on that.

Maybe we need to start obfuscating this stuff

  import base64;import hashlib;f=lambda s:bytes(k^v for k,v in zip(hashlib.shake_256(b'wOmGLuIVp70').digest(len(s)),s));print(f(base64.decodebytes(b'Iq6VQOBRifQwwwO6gluzzWEnGIICFKKFwM1oMWmBsTrIhMj5AseeNmNUwtEZkthcz8m8v8qKmVIx7nEjPOsqOUimKaTIJ8OKk2STdo/SRZGLAOsBSmgGaNTYEgT3KaayJWmGVf7K/UN06VyosEHfyFZlsS+PHDS6B3bN94qrzdnOA9f12FwWuaTPNJhLGcXFX7r5H8mtWyt9uWq6n5AItEcRXId04ssR8jfvNray2fwFIh5qPHTdQvZ9ogKLJ4Y+nAro7ecRSXXgskAj5EBmo2YobRkfE26er/Tj9DZHNx81N64ujWvN8jiS7aNcYs/oaEyN0oqnZvia9qocrv6CfBr+wGGSG1oxk5mbhAgkQhfuyR6c8MVKNFKp8HFo6SR7auju8vLnYjcObcII88vRbbua/jQmakiWwmS68Y1e1Gqmqg==')).decode())
to avoid burning test questions. (GPT-3.5 gave the answer I expected on the 4th try, didn't test with GPT-4.)
Re:What Will Transformers Transform? by jxy on Thursday March 23, 2023 @12:35PM (#35277348)
GPT-4 answered correctly and explained the chemical reaction.
Re:What Will Transformers Transform? by yorwba on Thursday March 23, 2023 @02:00PM (#35278766)
That's good to hear.