This is a distinction without a difference. The point is that putting HTML and JS logic in the same file is what was usually done in the early days of programming on the web, before separating code from event attributes into their own script tags and files became the norm.
All JSX does is the exact same mix in the opposite direction: it's not about JS inside HTML; it's HTML inside JS. And I understand that that's fine for many people -- and I've learned to stop caring -- but I take issue with pretending that it's a "paradigm shift".
Maybe your point is that you don't like the style of building up HTML strings by composing function calls. If so, fair enough, but I would argue that the jury is very much out on the trade-offs between that approach vs. an interpreter for a template language. React's string-building API is not the only example of this style; for instance lisps seem to do it that way pretty often (probably because sexprs are a quite natural syntax for this). Personally, I find the string-building approach much easier to reason about. It's a more explicit and well-defined problem. Instead of parsing and augmenting an existing string, the implementation just needs to transform some parameters to an output string.
As I said, I'm not really a fan of JSX itself because I find it weird (and unnecessary), but I do like the simple string-building API it gets translated into.
Are you talking about JSX? If so, this doesn't ring true to me; it isn't a mixture of HTML and javascript, it's just javascript with an HTML-like macro syntax. I've personally always found that really weird, but it isn't a hodgepodge, it's just normal javascript function calls with a weird syntax.