j_chance

Ok this took some time to think about. I've always conflated entropy and randomness/data, but this is wrong. Say that i sample 512 bytes of data from an entropy source. That data is no longer entropic, because it's no longer uncertain. It's now information (assuming it can be used to inform about the entropy source). It may be random looking, but not entropic.

I think I see what you mean. Information and entropy are fundamental opposites, information cannot be entropy and entropy cannot be information. Information is certainty, entropy is uncertainty.

So with information theory, we measure entropy sources to get information, but it's disconnected from the reality of entropy in physical processes. Which is maybe fine? It seems like an abstraction that is 0 cost. Like if i roll a dice and i compare that to flipping a coin, the relative amount of information yielded is based on the number of distinct outcomes given some arbitrary constraints (what side is facing "up" for each object assuming the human operates the object fairly).

If we then do some reasoning using this relative difference in amount of information, the physical, thermodynamic reality of the human doing the action, the object moving through space, etc etc is arbitrarily stripped away, but with no impact to the soundness of the reasoning about the information (assuming the human has not rigged the system/the entropy source is sound). Sorta like algebra over a field. Any field can be used so long as the axioms of the algebra hold.

So it all comes down to soundness of entropy sources...?

(this has improved my understanding of information theory _significantly_ :pray:)

Thanks!

> Existential reality is not made up of information

> Potential and potential resolving (true entropy) is the underlying phenomenon not state (information).

I think this leads to the question "is information a function of perception". Knowledge of resolution of potential is information, and the properties of entropy should pass through IMO

What about when information is encoded into physical state? If I sample an entropy source and record each sample as magnetism on a disk, information becomes a physical phenomenon?

How can we be sure there are no non-human entities observing all resolutions of potential at any given time and then modifying reality (e.g. existing beyond our understanding). Also consider actors at future points in time that may observe and act upon information that formed in the past. e.g. it's easy to dismiss the information from a dinosaur dying thousands of years ago, until some random human discovers it's toe and triggers a full excavation.

I don't have an answer to this, I just don't feel confident dismissing information as not a physical phenomenon itself. It's certainly a real feedback mechanism, it's just unclear if an actor is required for it to be a phenomenon.

Solving `Ax=b` is easy if `x` can be any value. But if you bound `x` such that all entries must be small (e.g. only zeroes and ones) then the problem is hard: https://en.wikipedia.org/wiki/Short_integer_solution_problem

What you described would be closer to learning with errors (https://en.m.wikipedia.org/wiki/Learning_with_errors) combined with SIS/SVP. Learning with errors is based on the parity learning problem (https://en.m.wikipedia.org/wiki/Parity_learning) in machine learning, which I take as a positive sign of security.

Interestingly you can get information theoretically secure constructions using lattices by tuning the parameters. For example, if you make the `A` matrix large enough it becomes statistically unlikely that more than 1 solution exists (e.g. generate a random binary vector `x` and it's unlikely that an `x'` exists that solves for `b`).