cfgauss2718

Even if you could get around the payload and energy density (power autonomy) constraints, which are very stringent, this seems like a dead product. Sure, it’s a cool project for some controls engineers. But I can scarcely imagine the deafening noise this thing makes, and no doubt it would tend to kick up clouds of dust anywhere it went, which is hardly desirable from a health perspective. This kind of device would be a permanent irritant in almost any environment. Thankfully, buildings are already well equipped with things like elevators. I’ll keep my wheeled cart, thank you very much.

We know from the No Communication theorem that quantum entanglement does not transmit information between classical observers who have no role in preparing the initial state of the entangled elements. Why should one think that this No Communication theorem is applicable to the phenomenological description of brain behavior? It seems that the speed of nerve conduction is at odds with the time constants of brain synchronization. If we are to doubt that quantum entanglement plays a role in accounting for this discrepancy, then what are the assumptions that underlie this doubt, and are they reasonable?

This series by lecturer Frederic P. Schuller is astounding work of thorough mathematical exposition delivered with clarity and charisma. This body serves as an exemplary introduction and tutorial on mathematics spanning propositional logic, topology, multilinear algebra, and differential geometry, towards the fundamentals of Quantum Mechanics and Gauge theory.

In quantum field theory (standard model of particle physics), photons are the quanta of a field (naturally identified with the quantized electromagnetic field, or U(1) gauge field), which evolves in a nonlinear way that is coupled not only to itself, but also the electron field (as photons are the carrier of the electromagnetic interaction). It seems to me that the concept of “photon” (the particle) is one that is useful in some contexts (like modeling the possible interactions between two electrons in a Feynman diagram), but that the concept of photon is not a fundamental constituent of reality. Certainly, wave packets of the field can interact by way of superposition and thereby altering the space-time evolution of the field through its coupling constant.

I think are overstating the strength of the No Communication Theorem. It does not state that communication via quantum entanglement is impossible in all cases; the theorem is weaker. From the article you linked:

“Being only a sufficient condition there can be extra cases where communication is not allowed and there can be also cases where is still possible to communicate through the quantum channel encoding more than the classical information.”

Consider the quoted assumption (again from the article you shared)

“An important assumption going into the theorem is that neither Alice nor Bob is allowed, in any way, to affect the preparation of the initial state. If Alice were allowed to take part in the preparation of the initial state, it would be trivially easy for her to encode a message into it; thus neither Alice nor Bob participates in the preparation of the initial state.”

Now it’s not clear to me how to correctly map this assumption onto the papers context. Who are the observers? Are they a pair of neurons on the ends of the axon? Are they neighboring myelin sheaths? Is it fair to assert that these observers have no role in preparation of the initial quantum state? Seeing as the entangled photons in the paper are produced as the result of chemical reactions occurring within the confines of the myelin sheaths causing excitation of carbon atoms and subsequent radiation of infrared photons, do the particular assumptions of the No Communication Theorem (a thought experiment that translates those assumptions into definite mathematical propositions) make sense?

Where are we drawing the boundary between these classical observers and the quantum system? Are there even any classical observers at all?

Your yawning dismissal strikes me as an overconfident conclusion.

I love when grammatical mistakes become unintentional puns

Here’s maybe a useful example. Consider a scalar potential function F on R^3 that describes some nonlinear spring law. At a point p=(x,y,z), the differential dF can be thought of as a (1,0) tensor measuring the spring force. It acts on a particle at p moving with velocity v to give the instantaneous work of the particle on the spring dF(p)(v). Now, suppose that we want to know how this quantity changes when we vary the x coordinate. The x coordinate is also a function of p, we can represent its differential as dx, which is a co-vector(field). The quantity that captures this change can be thought of as a (1,1) tensor field, which is related to the stiffness of the spring potential in the x direction at each point p. In the usual undergraduate setting, this tensor field is given as the hessian of F, call this H. The action of this tensor looks like the product u^T H(p) v, where in our case, u^T = dx(p) = [1 0 0]. A good giveaway for when a “co-vector” appears in a tensor calculation is whenever there is a “row vector” in a matrix operation (most people identify “column” vectors with proper vectors). It’s helpful in this case that “row” rhymes with “co-“.

Can you provide some examples of important tensors in physics for which the underlying vector space is infinite dimensional? I’m most familiar with the setting of tensor fields on manifolds, in which case the vector bundle consists of finite dimensional vector spaces. Nevertheless, I suppose in the absence of a pseudo-Riemannian metric one lacks a natural isomorphism between vectors/dual vectors. Does this “bidual” distinction arise in that case as well?

Yes, the “multi linear map” definition is accessible to an undergraduate who has taken linear algebra. However, the more common meaning of tensor in physics, like the metric tensor of spacetime, requires some more sophisticated background to understand (differential geometry, Lie Groups come to mind).

Here here! Functions do not depend on your choice of coordinates, only the components of tensors do! I think this is why it’s important to keep covariance and contravariance in mind. While tensor(fields) do not depend on coordinates intrinsically, the way we represent them when doing calculations most certainly does, and this is usefully characterized by co/contravariance.

I think the point above is that in physics tensor is usually overloaded, and those practicing physicists when they speak of tensors are more often referring to tensor fields, and most often this is in a context with more geometric structure than is required by a tensor space in reference to a vector vector bundle. Typically they (physicists) are dealing with domains where the tensor space is in reference to the tangent bundle of a smooth manifold, with the prototypical example being the metric tensor(field) of space time in general relativity. Another prominent example may include tensor fields defined in reference to the tangent bundle of a group of gauge transformations, as in quantum electrodynamics, quantum chromodynamics, etc.

Obviously these things are not just useful to physics, but are indispensable, and so I think the assertion that only the definition of tensor that is useful to physics is the definition tensor=multilinear map is somewhat out of step. Perhaps it would be better to assert that the concept of multilinear map is essential to every useful definition of tensors in physics.

Very well, let’s just agree that in physics (r,s) tensors usually refer to sections of the tensor product of some fixed number of copies of the tangent bundle (r copies) and cotangent bundle (s copies) of a smooth manifold (almost always pseudo-Riemannian) and leave it there. Elementary!

Agreed, I can’t help but feel there is some overcompensation driving the style of writing. It was difficult to finish.

There are some interesting parallels to ideas in this article and IIT. The focus on parsimony in networks, and pruning connections that are redundant to reveal the minimum topology (and the underlying computation)is reminiscent of parts of IIT: I’m thinking of the computation of the maximally irreducible concept structure via searching for a network partition which minimizes the integrated cause-effect information in the system. Such redundant connections are necessarily severed by the partition.

Nothing about this research violates conservation of energy. The article as written is advocating using excess solar or wind energy as input to this CO2->CH4 conversion process (which is electrolysis based) so that some of that energy can be reused later by burning methane. Later, as in when the wind isn’t blowing or the sun is t shining.

On a glance, the Chichilinsky theorem assumption of smoothness for the mapping between voter preferences And the vote result (the relation phi) seems burdensome. For example, many people might be effectively summarized as single issue voters - the topological consequences of a typical definition of differentiation (calculus) would seem unjustified. The exercise of exploring this world may be interesting, but I’m not convinced of its utility to politics.

The multiverse may be a consequence of Everett’s theory, but the reality shattering nature of that consequence warrants scrutiny of the theory. Consider one of Xeno’s paradoxes. One may postulate that to go from A to B that first one passes through a midpoint C, and by naively applying induction (I.e. without knowing under what circumstances am infinite sum converges to a finite value), one concludes that a consequence is motion itself cannot exist.

However, the absurdity of this consequence should mot lead us to doubt the existence of motion, but quite the opposite, it hints to us that the either the postulate is false, or otherwise we are missing some essential knowledge that allows to determine the correct consequences of that postulate.

How is the answer “everything that can happen does happen, just in an alternate universe that is identical except for the outcome of one single measurement” a parsimonious answer to the measurement problem? To quote Sam Harris, many-worlds seems to be the least parsimonious concept ever produced by science.

Sympy is a poor tool to learn because it simply doesn’t scale to problems one most often encounters, even in schooling. Frankly, CAS are so general and unintelligent that problems with well known and elegant closed-form solutions, when presented to a system like sympy, result in an output which is often not even human readable - thousands of algebraic terms for instance to describe the equations of motion for a simple double pendulum.

Personally I have found that the best tools are 1) a firm grasp of elementary calculus (differential, series, and integral) and 2) exposure to simple numerical methods that apply to a broad range of problems.

Armed with this knowledge, in my opinion, Julia is a far superior language to python and its package ecosystem has a brighter future. Indeed, the language has been built with a focus on mathematical modeling and efficient numerical computation. It is a more natural starting point for those with an interest in mathematical modeling and engineering science, and will serve anyone who learns it better than python+numpy+sympy+scipy.

If one thinks of a metric as a “distance measure”, which is to say, how similar is some input x to the “feature” encoded by a layer f(x), and if this feature corresponds to some submanifold of the data, then naturally this manifold will have curvature and the distance measure will do better to account for this curvature. Then generally the metric (in this case, defining a connection on the data manifold) should encode this curvature and therefore is a local quantity. If one chooses a fixed metric, then implicitly the data manifold is being treated as a flat space - like a vector space - which generally it is not. My favorite example for this is the earth, a 2-sphere that is embedded in a higher dimensional space. The correct similarity measure between points is the length of a geodesic connecting those points. If instead one were to just take a flat map (choice of coordinates) and compare their Euclidean distance, it would only be a decent approximation of similarity if the points are already very close. This is like the flat earth fallacy.

Yes of course any positive definite matrix can be used as a metric on the corresponding Euclidean space - but that doesn’t mean it’s necessarily useful as a metric. Hence I think it’s useful to distinguish things which could be a metric (in that a metric can be constructed from them), versus things which when applied as a metric actually provide some benefit.

In particular, if we believe the manifold hypothesis, then one should expect a useful metric on features to be local and not static - the quantity W’W clearly does not depend on the inputs to the layer at inference time, and so is static.

This NFM is a curious quantity . It has the flavor of a metric on the space of inputs to that layer. However, the fact that W’W remains proportional to DfDf’ seems to be an obvious consequence of the very form of f… since Df is itself Ds’WW’Ds, then this should be expected under some assumptions (perhaps mild) on the statistics of Ds, no?

You raise a fair point, I do think that it’s important to understand how the properties of the data manifest in the least-squares solution to Ax=b. Without that, the only insights we have are from analysis, while we would be remiss to overlook the more fundamental theory, which is linear algebra. However, my suspicion is that the answer to these same questions but applied to nonlinear function approximators is probably not much different from the insights we have already gained in more basic systems. However, the overly broad title of the manuscript doesn’t seem to point toward those kinds of questions (specifically, things like “how do properties of the data manifold manifest in the weight tensors”) and I’m not sure that one should equate those things to “learning”.

Indeed, hopefully they can be diverted from interest in LLMs towards actual science, like the neuroscience which revealed the existence of said mirror neurons.

Your point is a salient one. It would be useful if we could provide guarantees/bounds on generalization, or representation power, or understand how brittle a model is to shifts in the data distributions. Are these questions of the kind that are answered in part by the authors? I haven’t read the manuscript, but the title doesn’t indicate this is the aim of the research, but it indicates an eye to something much broader and vague (“learning”).

I agree with your interpretation. There is something there to be learned for sure, but I’m doubtful whatever that thing is will be a breakthrough in machine learning or optimization, nor that it will come by applying the tools of analysis. The idea of “emergence” is interesting although vague and bordering on unscientific. Maybe complexity theory, graph theory, and information theory might provide some insights. But in the end, I would guess those insights impact will be limited to tricks that can be used to engineer marginally better architectures or marginally faster training methods.

I haven’t read the manuscript yet, and am not sure that I will. However I don’t agree with the question. Gradient descent, the properties of the loss function are the “how”. It seems like you want to know how some properties of the data are manifested in the network itself during/after training (what these properties are doesn’t seem to be something that people know they are looking for). Maybe that’s what the authors are interested in as well. If I could bet money in Vegas on the answer to that question, my bet would be in most cases that structures we may probe in the network and see in them correlations to aspects of the problem or task that we (as humans) can recognize, well very likely this will boil down to approximations of fundamental and eminently useful quantities like, say, approximate singular value decompositions of regions in the data manifold, or approximate eigenfunctions etc. I could see how these kind of empirical investigations are interesting, but what would their impact be? Another guess, that these investigations may lead to insights that help engineers design better architectures or incrementally improve training methods. But I think that’s about it - this type of research strikes me as engineering and application.

By minimizing a loss functional with respect to a bunch of numbers that amount to entries in matrices (or tensors, whatever) using an approximate hill climbing approach. I’m not sure what insights there are to be gained here, it doesn’t seem more exotic or interesting to me then asking “how does the pseudo inverse of A ‘learn’ to approximate the formula Ax=b?”. Maybe this seems reductive, but once you nail down what the loss functional is (often MSE loss for regression or diffusion models, cross entropy for classification, and many others) and perhaps the particulars of the model architecture (feed-forward vs recurrent, fully connected bits vs convolutions, encoder/decoders) then it’s unclear to me what is left for us to discover about how “learning” works beyond understanding old fundamental algorithms like Newton-Krylov for minimizing nonlinear functions (which subsumes basically all deep learning and goes well beyond). My gut tells me that the curious among you should spend more time learning about fundamentals of optimization than puzzling over some special (and probably non-existent) alchemy inherent in deep networks.

Am I the only person who saw this headline and thought WOW, 12 inch transistors!?