thinkzilla

c.f. the Berry paradox https://en.wikipedia.org/wiki/Berry_paradox

I'm under the impression that the majority of startups go nowhere, so the fact that they don't have this property is not itself a major statement about the field as a whole.

The Attention Is All You Need paper deserves a lot of credit, but it also builds on other ideas. Earlier versions of attention were introduced in 2014 papers from (not U.S.) universities.

Making estimates is fine, but we should be clear that they are (provably) just opinions.

http://scribblethink.org/Work/kcsest.pdf

https://news.ycombinator.com/item?id=13731975

https://www.amazon.ca/Limits-Software-People-Projects-Perspe...

I generally agree with the point of the article ("Fourier transform is not magical").

However saying it is "just" curve fitting with sinusoids fails to mention that, among an infinite number of basis functions, there are some with useful properties, and sinusoids are one such: they are eigenvectors of shift-invariant linear systems (and hence are also eigenvectors of derivative operators).

The comparison to railroad infrastructure is interesting.

I think the author is wrong on this point however: > Today’s tech just cannot do what will be required of it (AI shouldn’t be dispensing medication when it can’t even count to 7).

The failures of AI are thought provoking, and more so when considered together with other results where AI performs at near expert level on challenging benchmarks. However, perfect reasoning is hardly a requirement. Most humans are not particularly good at reasoning, and most jobs do not need it. Both humans and AI can use calculators and other tools. All that's needed is that the AI is more or less as good as a human, while requiring much less pay.

A good exercise to appreciate the current state of AI might be to ask AI to write an essay about this topic ("how much revenue is needed to justify current AI spend, and draw parallels to the dotcom boom and building the transcontinental railroad"). Try it with two different models, using the deep research mode. I expect the results would be humbling.

I am sympathetic to the motivations and argument in the article, but the analogy with bridge building is flawed.

In engineering, if your assumptions are correct and you apply the formulas correctly, the bridge will not fall.

This is _not_ true of software, since it suffers from mathematical incompleteness. Computation is isomorphic to mathematics, and, just as there is no way to objectively estimate how long it will take to prove a theorem, there is no way _objectively_ estimate program properties, even simple things like "will this program ever print the string "xx". The proofs are variations of the Halting problem.

http://scribblethink.org/Work/Softestim/kcsest.pdf

Writing software is analogous to discovering the equations of physics of a bridge (physics/math) rather than applying them (engineering).

While the post uses DPO to illustrate RL and RLHF, in fact DPO is an alternative to RLHF that does not use RL. See the abstract of the DPO paper https://arxiv.org/abs/2305.18290, and Figure 1 in the paper: "DPO optimizes for human preferences while avoiding reinforcement learning".

The confusion is understandable. The definition of RL in the Sutton/Barto book extends over two chapters iirc, and after reading it I did not see how it differed from other learning methods. Studying some of the academic papers cleared things up.