Hmm it's a bit more subtle than that. There are all kinds of ways to misrepresent progress, the simplest way is to pick bogus metrics. For example, all of machine-learning based dialog system work uses metrics that were designed for machine translation ( basically n-gram overlaps ), this is an example of a metric that you can improve on but is meaningless. So the resulting system is very much the dog-and-pony show variety. Or people who use simulations to train and test ML systems, and say the metrics get better but do not address the sim2real gap.

> I asked an Ivy League professor about hype and they said the exact opposite of your Ivy League professor

The context matters, I have heard the opposite response as well, from a professor I was working for, while he was doing just that: packaging narratives w/o regard to how well the methods actually work. The honest take came at a wine-and-cheese party from a professor I was not working for. There is a real conflict of interest here, and if people stand to gain by withholding information, they will do Just that.

Just curious but what's the exact opposite of those statements? That he hasn't had to convince anyone of their work in order to get money? Or that it isn't based on that but on results? As someone that doesn't live in that world, I have no idea what the "exact opposite" of that really is.

For research funding, the sweet spot is the flashiest idea that seems extremely likely to work.

It’s hard to get grants to try something really out there, especially from the NIH or NSF; DARPA is a bit more adventurous. Pilot data helps with this, but those experiments cost money too, so one approach is to make the project sound almost, but not quite, obvious in light of existing work.

At the same time, it’s also really hard to get funding to confirm something that everybody “knows”, even when the underlying data is actually not very convincing. These experiments need some hype.