inbfour
Correct, but this feels tautological, or semantic.
In a closed environment the black box is the model, whether that be a single instance, or a mixture of 8 experts.
I wonder what that would look like?
Something like: if every best next token has a confidence below some threshold for some length of context then the model can confidently assume ignorance?
I would love to read the training corpus required to achieve this without metamodeling.
That would likely be a utopian forum community devoid of ego and gaslighting and filled with humble people with a thirst for knowledge who clearly communicate the limits of their current experience.
Think back to when these models were first released, or even back to v2 and v3. The models would have just “tried” to answer and it would have likely been a mangled mess.
But this version gave an erudite answer to an unasked question.
If the original prompt was “Can one write a password generator in INTERCAL?” I might agree with your claim of “johnsoning”, but it instead just said “Please write…”.
I think this points to the fact that the model is filtering its own answers, similar to if you ask it be offensive, or do your homework, or quote psychadaelic scifi.
This was what I wanted to highlight. That the model has been specifically and intentionally productized to lie to save the parent company’s coffers.
Responding with ~“I would but it’s just too complex” would satisfy most without the accompanying article, but the author is well trained and so provides a solution that would even fit within the model’s current meager token limits.
We are in agreement.
I would argue the threat model of cracking a psuedorandom string of 16 characters would probably want to start with a copy of the rng implementation used, but still the question remains what is the least “bullshit” way to respond?
You seem to want critical guidance, a la “INTERCAL as your choice of language may make your code vulnerable”, but INTERCAL was the ask.
That’s what the author wanted, and the author left out any explanation for the generator’s intended use, or cryptographic requirements.
Rather than appear to be advocating for using this INTERCAL “password generator” for any real world use, my point was that I think the solution the author produced should be a capability of these models.
For sake of clarity, it is my opinion that if you want cryptographic security you should use libsodium, but if you want to watch these models flounder use an adhoc rng implemented in INTERCAL.
1) These models will lie to obfuscate their incompetence.
Rather than say “I am unable to do this because my training data clearly lacked enough INTERCAL code to imitate”, the model waxed poetic about the odds stacked against it(i). Strange messaging.
The author’s implementation was less than 50 sparse lines of code that I am unable to speak to the complexity of, because my neural history lacks sufficient INTERCAL training.
I thought quick solutions to complex problems was the pitch?
What is the intended message of the models response? “I only provide solutions to superuncomplex problems”?
2) These models always highroad veiled communication.
Although I disagree with the author’s hypothesis that “spurring”(ii) is constructive, I do think it was intended merely as a quippy reference to “nerd sniping”.
Which presents its own interesting question: can these models be manipulated using reverse psychology?
Is mimicry of our vulnerabilities the imitation game?
(i) “however it would be quite complex and not straightforward”
(ii) “responds so politely to the type of shade that might have spurred a more human programmer into action.“