I understand one aspect of it in terms of probability. However another aspect of it eludes me as well, as I described in my last paragraph.

There's also a third angle from information theory. This type of understanding I haven't really studied in depth yet.