Just because it's confident, doesn't mean it's right. But it probably contains training data from this or similar sites.

https://enigma.hoerenberg.com/index.php?cat=The%20U534%20mes...

> Interpretation (preliminary):

> [An] U-4701, nachrichtlich [an] U-Stützpunkt Lübeck von Chef 4. U-Flottille: Mit U-4702 und U-4703 zur Flender Werft Lübeck gehen. Von dort folgt Weiteres.

> Translation (preliminary):

> [To] U-4701, for information [to] Submarine Base Lübeck from Chief of 4th Submarine Flotilla: With U-4702 and U-4703 go to Flender Dockyard at Lübeck. From there more follows.

There are about 2^67 different Enigma machine initial settings. The inverse probability of the appearance of a real seven letter German word (LUEBECK) twice in this message is a number that's pretty close to 2^67. So if you decrypted a message with incorrect settings you might expect to see those two appearances of LUEBECK, or a similarly misleading occurence. The presence of even a few of the other recognizable German words (or common abbreviations such as triple letters and the shortened names for the numbers) makes it overwhelmingly likely that this is the correct plaintext.

That would be true if it actually did the decryption, but my point was, an LLM doesn't decrypt. It just has the encrypted string followed by the decrypted string in its training data and so it outputs something that's almost correct. (the numbers being wrong 4501, 4502, 4503 instead of 4701, 4702, 4703 - maybe some bugged training data, maybe hallucination).

Sure, and I said it was confident, not correct.

I find it interesting that it pulled some kind of interpretation from the string. Far more than I would have. I asked it to translate the English data back into a similarly plaintext string and then asked a second instance to decode it and it came back with a similar, slightly distorted response.

The point is more to say that a language model is exactly the sort of thing that would be used to determine whether a given potentially decoded plaintext string is actually decoded, and given various anachronisms and shorthands our personal language models may not be adequate.

But a giant one that's been fed all sorts of data including examples of text of similar usage sounds actually like it might be exactly the tool for this problem.

Seems mostly reasonable, except for consistently translating SIBEN to fünf (five) rather than sieben (seven).