One of the first "hacker" breakthroughs in my first startup was due to following citations on a paper kind of related to my problem until I got to a paper from '61 implementing the exact algorithm I was needing at the time. Numerical approximations to a gnarly probabilistic problem that would make any engineer proud. Worked like a charm and, in a hardware thousands of times faster than it was designed to run, it was crazy efficient.

Could you please share the paper reference? Thank you.

One of the nicest things about finding old papers is that often the "unworkable" technique now fits in L1 or L2 cache.

Or that RAM is practically endless now. I still remember when I was reading an old pattern recognition paper and they dismissed one simple, elegant solution as "obviously impossible" because they estimated it would need 10485760 KB of RAM. We then bought a server with 128 GB of RAM and shipped their impossible approach in production :) where it's been running nicely for 9 years now.

Nice, hope you let them know.

Is there a name for this phenomenon, where with limited resources humans get more creative? There are still some astonishing technologies that are ongoing today, but much more limited to the early era of computer science.

Limitivity

Worse computers require better programmers.

Seriously. Ever try to pass an interview with "Well first I'd probably look up the optimal answer..."? It's worked for me about zero times.

Yeah, works about as well as "well, I'd see how the rest of your codebase does it".

What's really remarkable is that so much of the last decade or so of research has been grounded in work done in the 1970s and 1980s, when computational boundaries were narrow enough that the work was often largely speculative or significantly restricted in application, so it's only now that we're really seeing that work bear fruit. It's a pleasure reading old comp-sci papers, and I wish more developers took the time.

“Solving for colour constancy using a constrained dichromatic reflection model” is a good one to check out

I would love to see a list of "oldies but goldies" in various fields. Wonder if you're aware of such a thing?

https://people.math.harvard.edu/~ctm/home/text/others/shanno... would be a natural starting point.

There's a nifty use for AI: Read, grok, simulate, report, repeat ...

Such a bad idea, eh?

> So Carmack ... started reading research papers.

I feel it was more common in the earlier days of programming. With no open source "game engines" and the Githubbed, Stackoverflown internet we have now fetching about for some of the "programming secrets" whether that was in a university library or a technical computer magazine was about all you had.

Most of my career as an algorithm designer has been to boil problems down to their abstract essence, categorizing what kind of problem it is, reading papers to find a solution and implementing it.

Unfortunately, ACM and IEEE are paywalled, as are many of the specialty journals for operations research and the like. Newer stuff is often on Arxiv, but the simple algorithms usually live in older papers that aren't.

And this is the power of ChatGPT. Instead of reading a ton of literature, you can ask it for algorithms in literature that map to your problem.

I work in a space where I simply don't have bandwidth to read every paper out there, but a few question on ChatGPT managed to narrow the field for me in 30 seconds, and I could take it from there. It could even prototype the algorithm tailored to my specs. Game changer.

And it will give you a crap answer 95% of the time. Good luck using chatjippity when you have any level of complexity.

I’m working on a React UI based SQL query builder, where the entire data structure is recursive.

ChatGPT is literally unusable for this

That's a very biased point of view. I use it for mathematical formulations for optimization problems and it gives me very good answers most of the time. The literature is very strong but hard to search via keywords. ChatGPT manages to digest the literature and does a good job pattern matching to the right problem.

(I also use the paid version).

> UI based SQL query builder

Well, there’s your problem.

We're talking about finding references or details of existing algorithms to solve a specific problem. What you are trying to do has nothing to do with that and it's a waste of time with an LLM.

> And it will give you a crap answer 95% of the time

You're underselling it. Not only will it give you a crap answer, but it'll give it to you with unerring confidence!

> When I moved out of computing into biochemistry and medicine, and now into climate, I found people take papers seriously, which is a pleasure.

SIGGRAPH had been around since 1974 and was well-known if you were doing anything graphics related--especially 3D. You couldn't help but trip over the algorithms you needed.

The genius was putting any of these algorithms meant for super-expensive 3D graphics hardware on a wimpy-ass 1990 PC.

You're reminding of the "Numerical Recipes" series of books from that era. I feel like there were a series related to computer graphics as well....

Numerical Recipes was just several editions of one book, each with a few variants (for different programming languages).

But you might be thinking of Graphics Gems (vols 1–5). https://www.realtimerendering.com/resources/GraphicsGems/

Abrash's Black Book was more in the style of Numerical Recipes than graphics gems, from what I recall. I also remember Nvidia putting out something annually around the late 2000s that was pretty good, can't remember the name though.

Wow, yeah, Graphics Gems, those was amazing. Thanks for reminding me.

From the same era, folks may also enjoy the three volumes of collected articles from Jim Blinn's Corner. Some of it is a bit dated now, but plenty is still relevant.

Exactly. Research and it’s practical application is pretty common in computer graphics, and particularly cutting edge graphics engine development. There’s no need to dig around in academic papers unless you’re dealing with deep technical problems and off-the-shelf solutions are out of the question.

One of the things that drives me nuts about this industry is how ahistorical it is. The end result is that we keep reinventing the same things over and over especially around operating systems, databases, and languages.

At least in open source, I would claim that it's a result of volunteers doing the work for fun, which necessarily means they follow the dopamine: implementing someone else's work is not nearly as fun as figuring out a problem for yourself.

And the reading comprehension of CS people is near zero. You can give someone a paper with the solution, they will skim, not understand anything and then going back to reimplementing something poorly from first principles.

> reimplementing something poorly from first principles

Not-invented-here syndrome in practice. Which kinda works for the n-th shitty web framework. But it goes really poorly in any complex domain.

Are there any papers at all I. Web framework design?

This has nothing to do with open source. For GraphHopper I researched many papers regarding fast routing algorithms or advanced data structures etc ... and I'm pretty sure that this is true for other open source projects too.

It could be even the opposite: because many people can look into the code the code converges to the state of the art.

My theory is that open-source projects are a reflection of the language and all languages have the same flaw. Languages make it impossible to create simple objects with simple logic that can then evolve to more complex ability without modifying the simple object.

This results in new contributors being overwhelmed with the complexity of the classes and totally pushing aside the prior work in favor of starting at the basic level.

The module systems of common languages (like C# and Rust) tend to represent a "bag" of structs and traits, whereas a more advanced "evolutionary" language would represent source files or modules in a hierarchy of simple to advanced behavior.

Julia bucks the trend with its static duck typing; it's one of few languages where composition really works well. If your type defines + and * you can make matrices of them and multiply them together, then do linear algebra with them or format them as spreadsheets. Numpy, despite Python's duck typing, fails here because it's a thin wrapper over C, so the types are merely proxies and not extensible. Julia's compiled and heavily optimized, so it lives up to the promise. Also, unlike most static languages, the type system looks at the body of functions to infer what types they'll support - everything is maximally generic by default.

Julia has many flaws, but they're a different kind of flaws. Mostly caused by choosing function overloads based on vibes, and reckoning with the pandemonium caused by modules being glued together in unanticipated ways.

will look into this, thanks.

The industry is fundamentally conservative and un-imaginative. That being the case the M.O. is always a slightly tiny spin on an existing thing or a recreation of something that has been around forever but is currently slightly less popular then it once was.

> most of those papers write-once-read-never

Does someone know a good way how to put a filter on these "write-once-read-never" papers and just get the top 10 papers each month?

Fully agree. There's so many people who succumb to "not invented here" and then run blindly into solved issues.

I keep repeating the story, but in 2020, I surprisingly got #1 on the MPI Sintel benchmark for optical flow in the category "Clean EPE matched", by re-implementing a 15 year old paper. (And replacing their approximation of a quadratic solver with actually using a quadratic solver.)

That said, I feel like it's not just machine learning which is flooded with useless preprints. It's the entire internet now. When I wanted to look up some context for quaternion math, it was surprisingly challenging to find something a bit more in-depth than motivated amateurs who try to sell their self-published ebook and/or Udemy course. No, "np.quaternion" is not a mathematical formula. In the end, it was easier to dig up my old paper book, which had a nice A4 overview page with all the relevant formulas.

The disconnect between laypeople and research is often compounded by the stilted language, undefined terminology and related knowledge assumptions, underspecified details, and incomplete presentation of code or examples for independent reproducibility.

Research papers, to be maximally reusable and future-proof, should prefer to present information in as plain English as possible and commonly-used mathematic terminology and symbols rather than obscure forms where possible. Using big words (or Latin) doesn't add profoundness, but they can add artistic flair in other contexts where elucidation isn't the goal. ;-)

Would using standard Basic English (https://en.wikipedia.org/wiki/Basic_English) help?

And source code, for the love of god, give me source code so I don't have to guess what your underspecified description in the paper actually means.

>in as plain English as possible

This is exactly what they do, even domain specific language is selected to precisely convey meaning and differentiate from incorrect but close interpretations.

We must be reading very different papers.

My experience has been that the older CS papers (from the 60s, 70s, 80s) commonly have very clear, readable text, not weighed down with unnecessary "academic style" verbiage. It's the newer stuff that's painful to read.

I don't know whether this is a general change in fashion, or because the early papers in a field are dealing with simpler ideas (at least ideas that are not reliant on a decade of prior papers and terminology), or because it's only the good old papers that still come up in literature reviews and all the badly written stuff has long since been forgotten.

As a researcher I can say the opposite. In my field (cryptography) modern papers usually start by specifying formally exactly what the algorithm APIs look like; then they provide precise security definitions; then they describe algorithms; then they provide formal (if detailed) proofs of security. It’s long and detailed, but you get all the pieces you need for precision and you know where to find it.

Papers in the same conferences from 1982 are full of beautiful insights and they’re easy to read. But often that’s because they’re a five-page long story that might provide only a tiny subset of what we expect nowadays. They often don’t specify exactly what the algorithms do, they just kind of describe the ideas in English. Formal definitions are like three sentences of handwave. Proofs are similarly short or missing. Typically if you follow the citation chain you’ll find a dozen papers saying “we show that the original construction by Seminal Author is totally broken in this situation, but we show how to fix that with a small tweak and a better security definition.” Eventually you get to modern papers that actually start with the right definitions to deal with all that stuff, but take longer to read.

Obviously to the layperson the short old papers are fun and full of insight! To an expert they’re still full of insight, but we wouldn’t let them out of our lab in that condition. It’s very much like comparing a beautiful old classic car to a modern car with safety features. They both look great: but you don’t want to get into even a minor fender bender in the old car.

The constant fighting between Schmidhuber and LeCunn should be well known in here. But in my opinion, both of them have been involved with fantastic papers that are both painless to read and full of useful info:

CTC loss: https://www.cs.toronto.edu/~graves/icml_2006.pdf

DeepLearning: https://hal.science/hal-04206682/document

And both of these papers are new and on more recent research, so it's not just old papers that are good. It's just that there is a flood of bad new papers drowning out a constant signal with additional noise.

> Research papers, to be maximally reusable and future-proof, should prefer to present information in as plain English as possible and commonly-used mathematic terminology and symbols rather than obscure forms where possible. Using big words (or Latin) doesn't add profoundness, but they can add artistic flair in other contexts where elucidation isn't the goal. ;-)

As someone with a background in academia, I have to disagree with the assertion that papers should be written in "plain English" and some lowest-common-denominator of symbolism.

It doesn't make me happy to argue against that, but the truth is that scientific papers MUST build on previous knowledge. No leading gene editing research paper is coming out with long-winded prose explaining what DNA and RNA are. No particle physics paper coming out of CERN is going to avoid quantum field theory jargon and terminology in the hopes that a lay person with a pre-calculus level of math education is going to be able to study the results and understand it. How many times longer would these papers be if they did try to do that? My calculus book from undergrad was some 600 pages long (it covered three semesters, to be fair)--now imagine that the people writing these papers read that 600 page book in their first one or two years of undergrad, and then proceeded to study their subject for AT LEAST another 6 years (closer to 8 for physics) before getting "Dr" in front of their name. How many pages would a research paper need to be for a lay person to understand? Probably pretty long.

The truth is that there sometimes IS NOT a "plain English" way to explain something. Yes, you can use inaccurate metaphors and analogies about cats in boxes, but that doesn't lead to someone actually understanding the Schrodinger equation and Hilbert spaces. Believe it or not, scientists are not sitting around all day figuring out which Latin words will sound the most "profound" in their prose. Those are just the words they really actually use. Because words have meaning and some ideas are so common that they deserve to be condensed into a word, or an equation, or a symbol...

At the end of the day, these papers are written to advance the respective field of study. The best way to do that is to write papers that are going to be relevant to people who might actually build on the results. The people who might actually build on the results are going to be people who understand the requisite background information. If Joe Schmoe wants to read a particle physics paper, then he better pull out his high school math books and work his way up to at least a masters degree level understanding of physics and mathematics.

I would never go into the kitchen of a professional chef and tell her to stop referring to a "paring knife" or a "fillet knife" because I'm not familiar with cooking jargon. I wouldn't tell her that she has to re-explain what shape those knives have every time she explains a new recipe or technique. I would just have to learn the jargon if I actually cared enough.

if it weren't for NIH, i'd bet that a majority of HN readers would be unemployed right now!

Writing code is more fun than reading. Trying to solve problems is more fun than researching previous problems. It takes a special kind of discipline to get to the point where your first instinct is "Let me book out a few days and survey the literature" instead of "Let me use all of my IQ points to try to implement an ad hoc, informally-specified, bug-ridden, slow implementation of half of $ANYTHING".

I never had this impression. For me reading papers is way more fun and easier than developing a new solution in code and painstakingly work over all the bugs until they are gone, or worse, find bugs that you don't have solutions for. Or finding that it is too slow for your target machines.

Creating something new is so much work.

Like skying it could be fun when you are not a professional and you do it on your terms, but once you are a professional it is something different.

In truth, I'm much more this way myself than the median dev. Unfortunately it's not the greatest strategy for climbing the ranks and getting yourself to a top paying senior engineering gig to be a talented bug-fixer, and never a talented full program creator (or wheel-reinventor).

Nobody mentioned this but software patents are/were a pretty good reason to avoid looking at other people's algorithms.

In...the countries where that was a problem.

My favorite tech has a handful of patents expired in the past two years. But they are written in a way that doesn't expose the underlying algorithms, just the "steps" which seems to be a very keen well-designed patent informed by lawyers. Designed to the letter of the law. Mitsubisi Research Laboratories funded the work, it's clear they wanted to protect their investment. The text inside the patent is this absurd abstraction of the algorithm such that it could apply to anything.

Software patents exist in this space where the patent holder gets the best of both worlds: they get to argue that an infringer nebulously infringes on the ambiguous "steps". Meanwhile the source code is not in the patent, therefore protected from public eyes. Patents should require exposing the source code.

Back in the day access to knowledge was quite hard, everyone first started with random file dumps from BBS exchanges, then followed by books. After that if you were of the right age and lived in the right place you might have had access to a university library with papers.

Talking of knowledge and books: It was around 1995 or so and I was in my last year of high school. We went on a small field trip to a local computer show and only about 4-5 students in the whole school were interested in going, at that time computer science was still a very new topic over here in the 3rd world.

At the show I found a stall in a corner with some random hardware books and then this giant white tome about computer graphics. The size of it was intimidating and while scanning through it I saw some ridiculous mathematical formulas that I couldn't make head or tails of. But either way I had to have it and luckily I had gotten just enough money from my mom that I was able to buy it.

That was how I lucked into Computer Graphics - Principles and Practice (2nd Edition) which had just come out. That book was a literal gold mine of concentrated topics for a kid, I spent more time studying that book in the next year than I had done for all my high school subjects combined.

My deepest thanks to everyone who contributed to it, it's shown here on Fabien's page: https://fabiensanglard.net/Computer_Graphics_Principles_and_...

Not to ramble, but a funny addition to the story. The only compilers/languages I could find locally were DOS Basic and then Pascal with Assembly which I learned most of my programming skills in. I had heard of this magical C language that was so fast and all the games were now being made in it, but for probably 3 years I couldn't find a compiler for it anywhere.

Then randomly one day right around the Doom 1994 release date my uncle who had started working with hardware had been given a promotional Watcom C/C++ compiler which he gifted to me, and I could suddenly work with the same language and compiler that they had used! Of course it took me a bit of time to learn but by the time I got the CG Bible in 1995 I was set.

Wasn't GCC already out in 1995? (Though needing an internet connection to download it was an issue)

I blame agile and SaaS, neither of which are super tolerant of developers taking a few days just to read up on things. When you're both an owner of the company and/or just need to ship one good version of something, you get a little more space to do it right.

WriteOnly