chas
There are chemical systems where Lyapunov time is small enough that you can only predict seconds or minutes into the future and astronomical systems that are nonlinear and chaotic but have a long enough Lyapunov time that you can make reasonable predictions for millions of years. For both of these scales, the Lyapunov time still bounds how far into the future you can expect your predictions to remain near to the actual behavior of the system.
I do not know the Lyapunov times of the financial markets. That said, mathematically-sophisticated professional analysis frequently get their predictions wrong in major ways, so I expect there is a pretty hard bound on predictive quality caused by a short Lyanpunov time of the markets themselves.
This means that people who have an up-to-date knowledge of a given subfield can quickly get a lot out of a new papers. Unfortunately, it also means that it usually takes a pretty decent stack of papers to get up to speed on a new subfield since you have to read the important segments of the commonly cited papers in order to gain the common knowledge that papers are being diffed against.
Traditionally, this issue is solved by textbooks, since the base set of ideas in a given field or subfield is pretty stable. ML has been moving pretty fast in recent years, so there is still a sizable gap between the base knowledge required for productive paper reading and what you can get out of a textbook. For example, Goodfellow et al [1] is a great intro to the core ideas of deep learning, but it was published before transformers were invented, so it doesn’t mention them at all.
That said, we are discussing this in the context of walking a bit more, which should be very well-tolerated for pretty much everyone as long as they don’t have other significant health problems and work up gradually. For example, if you take 1000 steps a day, space out the increase by each week increasing your daily steps by 500-1000 steps a day until you reach 10000 steps a day over the course of a few months (temporarily pause or reverse the increases if you see anything other than passing muscular discomfort).
This also explains how ultramarathon (or other “extreme” physical activities) can be tolerated without much issue—bodies are very good at adapting to the loads that are placed on them as long as that increase is gradual and they have enough food and rest to rebuild after particularly intense bouts of activity.
A further corollary is that over the long term, being sedentary is not well-tolerated at all, so maintaining a certain baseline level of physical activity is definitely a better idea and as long as your body isn’t giving you any feedback to the contrary, you shouldn’t avoid high loads or intense activity just out of abstract fear of injury.
On the other hand, regular expressions and things like lexers are largely based on finite-state machines so there is a ton of information related to those, but they often implement things more complex than finite-state machines in order to be more expressive.
In terms of manually implementing them yourself, I think the naive implementation based on the mathematical definition of a deterministic finite automata is pretty good for a small number of states for things like tracking program state. In particular, explicitly listing the total set of expected inputs/transition criteria, explicitly listing the states, and having a single function that takes the current state and current transition-relevant data and produces a new state.
This representation is nice because it makes the behavior inspectable in one location. This makes it easier to notice the edge cases and prevent the state representation and possible transitions from getting spread all over code. The transition function can be a switch statement or a table. Really any way of writing a two-input function with a finite number of inputs and outputs will work. Many people have also explored strongly-typed versions of this, which are worth a look as well.
Since pure functions compute the same results under lazy or strict evaluation and require that any data dependencies they have are explicitly provided as inputs, they interact with lazy computations in a much more tractable way. This means that adding a strictness operator to a lazy language is much easier than adding a laziness operator a a strict language.
An alternate approach is what python did with generators where there is a data type for lazy computation, but it lives apart from the rest of the language, so it is mostly used for e.g. stream processing where a default-lazy approach is conceptually straightforward and is less likely to lead to extremely non-trivial control flow. This approach does, however, basically give up on having a laziness operator that will turn a strict computation into a lazy one.
A lot of the code I write is looping over all of the values of a data structure to either look for a certain condition being true (any/all boolean monoid), look for an extremal value (min/max semigroup), or combine all of the values in the data structure (sum/product/average monoid). The product of a monoid is also a monoid, so if you want to do two or more of those operations at once, the code remains simple and orthogonal while only traversing the structure once.
In particular, all of the those operations are nicely expressed as one-lines using foldMap (https://hackage.haskell.org/package/base-4.9.1.0/docs/Data-F...). If you have a data structure called `xs` and each element in data structure has an integer field called `size` you can do the following by using foldMap with different Monoid instances.
Sum the sizes:
let (Sum total_size) = foldMap (\x -> Sum (size x)) xs
let (Any over_threshold) = foldMap (\x -> Any ((size x) > 5)) xs
let (Sum total_size, Any over_threshold) = foldMap (\x -> (Sum (size x), Any ((size x) > 5))) xs
let (First over_threshold) = foldMap (\x -> First (if (size x) > 5 then Just x else Nothing)) xs
let over_threshold = foldMap (\x -> if (size x) > 5 then [x] else []) xs
def sum(xs):
if xs == []:
return 0
else:
x = xs.pop(0)
return x + sum(xs)
While I covered the case of things that are sort of like reduce (called catamorphisms in the language of recursion schemes), this paper also has analogous abstractions for things that are sort of like itertools.accumulate (https://docs.python.org/3/library/itertools.html#itertools.a..., called anamorphisms in recursion schemes), as well as combinations thereof (called hylomorphisms). They all use a relatively small number of building blocks and combine in a quite beautiful and useful way, but it's hard to describe precisely without leaning quite heavily on the language of strongly-typed functional programming.
Do you have a favorite example that highlights the unique computational properties of vector spaces?
*I don't know how this changes in the finitely-presented case, but I assume the extra constraint can be used to improve the performance of the algorithms. It's a lot easier to find asymptotic analysis of the finitely-generated case though and I don't see a way around dealing with the fact that it's still not free.
[0] - I'm basing this on Chapter 8 of https://cs.uwaterloo.ca/~astorjoh/diss2up.pdf, but this is a deep field in which I am not an expert, so if you are, I'd love to hear more.
That said, I would be surprised if Excel spreadsheets were implemented as matrices. Since you can update one cell and have it automatically update any computation that uses that cell, I would expect spreadsheets to be implemented with some sort of dependency graph so it’s easy to traverse and update the values that need to be changed. (This could be implemented as an adjacency matrix, but I haven’t seen that representation used before for programming language dataflow analysis.)
Since all linear transforms between vector spaces with a finite basis are finite matrices, the computational tools make it tractable to calculate properties of vector spaces that aren’t even decidable for e.g. groups. For a simple, but remarkable example: All finite vector spaces of the same dimension are isomorphic, but in general, it’s undecidable to compute if two finitely-presented groups are isomorphic.
While having a more restorative life outside your job will help, I also agree that a therapist is a good idea to help with your anxiety and to help while navigating the transition from coding being your entire life to it being part of your life (at least for now).
To contribute to the OP, https://godbolt.org/ is indispensible for high-performance C++ development. In specific, you can see under what circumstances the compiler optimizations apply to your code.
If you are dead set on learning about monads without learning Haskell (or a related typed functional language), I would focus on understanding Haskell-style functors and then applicative functors because they are simpler than monads, but they are deeply-related algebraic building blocks that are useful abstractions in their own right. When you are comfortable programming with them, monads shouldn’t be much of a leap.
While floating point numbers can be cleanly split into a mantissa and exponent, adding floats requires shifting the exponent, which can be an expensive operation. Each portion of a floating point arithmetic operation is also implemented with integer arithmetic, which limits the performance spread. Many floating point operations require significantly fewer bits though, which can lead to major speedups.
On the integer side, the serial carry dependence mean ripple-carry adders are usually a bummer, but there a ton of carry-lookahead variants that can lessen the performance impact of that dependency. If you have several integer additions at once, you can use a carry-save variant to only pay that serial cost once all of the additions are done. Finally, if you are willing to significantly change your integer encoding, there are tools like redundant number systems[0] that allow you to move around the traditional trade-offs for arithmetic circuits including completely removing the serial dependency on carries. That said, if you require rescaling your fixed-point numbers during the computation, the fixed-point implementation will require more integer operations than floating-point operations.
All of this is also quite dependent on the overall architecture of the chip too, since the number of integer units vs float units and how data is moved around can have a way bigger impact on performance than how each arithmetic operation is implemented.
While I thought I'd be focusing 100% on discrete math in my computing life, I was really happy to have been exposed to gradients and convolutions when neural networks came to prominence. I've also recently gotten interested in power electronics and have been really grateful for having previously been exposed to several varieties of transistors and having had half a semester of thermodynamics. Not to say that this background was really enough to be productive, but it was really nice to already know that these subjects exist and be familiar enough with their broad outlines to not have to completely bootstrap multiple fields at the same time.
I'm a little salty about this though because, at the time, there were a ton of classes that I thought were boring and useless. I was wrong (at least about the uselessness) about for vast majority of them.