torustic
0. Milewski's "Category Theory for Programmers"[0]
1. Goldblatt's "Topoi"[1]
2. McLarty's "The Uses and Abuses of the History of Topos Theory"[2] (this does not require [1], it just undoes some historical assumptions made in [1] and, like everything else by McLarty, is extraordinarily well-written)
3. Goldblatt's "Lectures on the Hyperreals"[3]
4. Nelson's "Radically Elementary Probability Theory"[4]
5. Tao's "Ultraproducts as a Bridge Between Discrete and Continuous Analysis"[5]
6. Some canonical machine learning text, like Murphy[6] or Bishop[7]
7. Koller/Friedman's "Probabilistic Graphical Models"[8]
8. Lawvere's "Taking Categories Seriously"[9]
From there you should see a variety of paths for mapping (things:Uncertainty) <-> (things:Structure). The Giry monad is just one of them, and would probably be understandable after reading Barr/Wells' "Toposes, Triples and Theories"[10].
The above list also assumes some comfort with integration. Particularly good books in line with this pedagogical path might be:
9. Any and all canonical intros to real analysis
10. Malliavin's "Integration and Probability"[11]
11. Segal/Kunze's "Integrals and Operators"[12]
Similarly, some normative focus on probability would be useful:
12. Jaynes' "Probability Theory"[13]
13. Pearl's "Causality"[14]
---
[0] https://bartoszmilewski.com/2014/10/28/category-theory-for-p...
[1] https://www.amazon.com/Topoi-Categorial-Analysis-Logic-Mathe...
[2] http://www.cwru.edu/artsci/phil/UsesandAbuses%20HistoryTopos...
[3] https://www.amazon.com/Lectures-Hyperreals-Introduction-Nons...
[4] https://web.math.princeton.edu/%7Enelson/books/rept.pdf
[5] https://www.youtube.com/watch?v=IS9fsr3yGLE
[6] https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...
[7] https://www.amazon.com/Pattern-Recognition-Learning-Informat...
[8] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...
[9] http://www.emis.de/journals/TAC/reprints/articles/8/tr8.pdf
[10] http://www.tac.mta.ca/tac/reprints/articles/12/tr12.pdf
[11] https://www.springer.com/us/book/9780387944098
[12] https://www.amazon.com/Integrals-Operators-Grundlehren-mathe...
[13] http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
[14] https://www.amazon.com/Causality-Reasoning-Inference-Judea-P...
See this post and its comments[0] to get some foundations upon which to think about these correspondences.
[0] https://golem.ph.utexas.edu/category/2014/10/where_do_probab...
> 50 pages of pure cake, beautifully and informally written. The book is 1000 pages long. There is some new cake there, but not 20 times as much.
[0] https://arxiv.org/abs/math/0306109
[1] https://mathematicswithoutapologies.wordpress.com/2015/05/13...
I both like and dislike theories of mind that describe the brain in terms of n-veiled solipsism. But I generally like them because of the specific (decision-theoretic) statements they make about how an agent is defining its environment.
What's striking to me about life on earth -- mostly humanity, but also the larger notions of noo-/biosphere -- is that, so far, it doesn't match up to our larger-scale evidence about how <things in general> define <their environments>.
We have managed to define our environment in a fashion which is scale-agnostic, at least with respect to the measurement of physical accumulations of stuff. We readily form metaphors between:
(a) things in our locale,
(b) things in the huge,
(c) things in the tiny, and
(d) things in the meta (read: math/pattern).
I think the worst thing we can do -- as engineers or scientists or mathematicians or even metaphysicians -- is to presume that any of |a-d| generate the others "in reality". (I just did a bit of that when I implicitly defined "larger-scale evidence" as "most legitimate evidence".)
In the categorical sense of arrow-reversal-without-structural-change[0], models of |a-d| are dual to any notion of 'environment' we can imagine. That doesn't suggest determinism, but it does offload every variation on the question of free will into a(n empty) simulationist argument whose limit is some computational infinity, like an omni-* deity or an extremely general structure like an ∞-topos[1].
An interesting partial exception is complexity theory. When we map out statements like PostBQP==PP[2], we're doing something fundamental, that bridges the gaps in |a-d| without adding something too weird as |e|. The actual notion of 'complexity-wise truth' that's being generated[3] is a measure of inferential distance that doesn't seem to care about pesky details but is somehow still attuned to specific problems.
There are caveats -- you could imagine each distinct problem/distance class as the dual of a scope (our syntactic presentation of such scope anyway[4]). The segregation of our environment into such scopes is only the first half of the problem. Less trivial is the act of mapping out our environment as a structured space of such scopes, so that we dissolve all questions of type ['what is', 'what should be', 'what was', etc.] into a singular question of type ['what direction to go now'].
To me, this sounds quite dangerous. And yet some variation on that is what many of us in the greater "self-directed tech" milieu are working on. There's an embedded urgency to the framework. Call it "the market" or "the simulation" or "the basilisk" or "bringing about the singularity" or "humans vs. ai" or "geopolitics" or whatever you want, but what's difficult to ignore is how effectively |bcd| equivalences eat up |a|, for any definition of |a| that invites historical norms.
Interpret this how you will.[5] For me it causes a sharp recoil from generalizations like "we are just the universe observing itself." The recoil is followed by a question, "okay what should I observe so that I might identify with it?" To which the answer is usually, "whatever makes me a better engineer, so that I can worry about this later, when I have a better understanding of what I'm asking." (i.e., I fail to prevent |bcd|>|a|, and each iteration on the theme speeds up the process a little bit.)
[0] https://ncatlab.org/nlab/show/duality
[1] I should say I feel pretty awful making this analogy, as it performs a |bc| equivalence at the expense of most of |a| and everything important in |d|. It's no better than "what if atoms are tiny universes?"-type questions, and it should be treated as illegitimate as evidence for anything more detailed than the notion of existence itself.
[2] http://blog.computationalcomplexity.org/2004/01/complexity-c...
[3] https://complexityzoo.uwaterloo.ca/Complexity_Zoo
[4] https://golem.ph.utexas.edu/category/2010/07/what_is_a_theor... (though this analogy is almost worse than [1]'s)
Minimum description length might be just the right place.
[0] https://en.wikipedia.org/wiki/Minimum_description_length
[1] https://www.amazon.com/Description-Principle-Adaptive-Comput...
Alternately, generating functions.
[2] https://en.wikipedia.org/wiki/Generating_function
[3] https://www.amazon.com/generatingfunctionology-Third-Herbert...
Alternately alternately, various categorical notions.
[4] https://en.wikipedia.org/wiki/Realizability
[5] http://stijnvermeeren.be/download/mathematics/essay.pdf
[6] https://ncatlab.org/nlab/show/locally+presentable+categories...
[7] https://ncatlab.org/nlab/show/groupoid+cardinality
Alternately alternately alternately, foundational madness of the best kind.
[8] http://www.forkinganddividing.com/
[9] https://www.amazon.com/Blind-Spot-Lectures-Logic/dp/30371908...
One of the nice things about mathematics is that we don't need to say anything like that.
Probabilities are nature's probability techniques. Functions are nature's function techniques. And compression is nature's set of compression techniques.
You might find category theory useful here.[0][1][2]
Take the category Set, which is the category of sets.[3] Just imagine an n-dimensional structure of dots and arrows, where the dots and arrows can each represent some other n-dimensional structure of dots and arrows. Set is the structure of that form which is constrained precisely so that it represents the idea of sets, and functions between them.
You can consider other structures like Set[4] that are constrained (or freed) to represent all manner of properties, structures and other stuff. Some of this stuff resembles branches of mathematics. Some resembles physics. Some resembles philosophy.
Science takes observation and uses mathematical models to transform that observation into prediction. So long as a model is sufficiently general, in a computational sense, then you will find that you _can_ rewrite the whole of science in terms of that model. But this doesn't consider the question of elegance -- and the related question of what you want to consider fundamental. And the product question of those two: how do you measure/compute most effectively to maximize elegance-of-stuff upon that fundamental-structure?
As someone who cares deeply about constructive mathematics, I consider things like topoi, the Curry-Howard(-Lambek) isomorphism[5], and Grothendieck's relative perspective[6] to be fundamental, and a beautiful foundation for the modelling of any and all information dynamics (≃ 'nature', 'thought', 'computing'). You may have some other perspective. There is a vast space of ways to approach the problem of modelling experience.
To be scientific, the thing to avoid is appending some _intention_ to the processes that your model describes. Causality is real and everywhere, but there is no way to say "nature compresses by using probability and functions" that doesn't beg the question. Either compression, probability and functions are part of your singular model of nature, or they are competing models. You do math -- or coming from different directions you do physics, info theory, etc. -- by finding a general method that reworks them as a coherent idea.
You're doing the first step of that when you form a surface analogy between (probability,functions) ≃ (nature,compression).
If this kind of question bothers you enough, you can make it your life's work to follow the rabbit hole all the way down. It's not so much a question of smartness (though rigor is a huge part of truth-seeking) -- just how inexorably bothered you are by the idea of competing models of reality.
Cryptography, machine learning, and complexity theory are immediate implications of analogies similar to your own (p,f)≃(n,c). Should you want to study further (and don't particularly identify with the constructivist tone of this comment), those are probably the most appropriate fields in which to look for elegant solutions to your questions.
[0] https://en.wikipedia.org/wiki/Category_theory
[1] https://ncatlab.org/nlab/show/nPOV
[2] https://arxiv.org/abs/math/0004133
[3] https://en.wikipedia.org/wiki/Category_of_sets
[4] https://en.wikipedia.org/wiki/Topos#Elementary_topoi_.28topo...
[5] https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspon...
[6] https://en.wikipedia.org/wiki/Grothendieck%27s_relative_poin...