Microsoft Research is essentially hoarding technology that goes substantially beyond mere `rr`. Especially when async/coroutine models are involved.

It's locked behind the really expensive (like 10k or so? For I guess a seat a year?) Visual studio license and IIUC windows-only.

Yes, I mentioned gdb's reversible debugging capability late in my comment. :)

My experience was that C++/Java debuggers beat interpreted language debuggers shockingly well. So much so I still don't touch interpreted language debuggers today. I should check in sometime on Python soon.

Debug code has to, essentially, include debug symbols and tracking, or, alternatively, there's that supervisor monitoring all source-code relevant locations. Things will be slower ... on a conventional x86 chip.

I would suggest that its plausible today that a separate _core_ could be developed for internal system monitoring (think on-SoC JTAG and you get the idea) and that core could be dedicated to more OS functionality otherwise. Notice that this involves a different ISA.

The core tooling advancement problem I believe reduces to this:

> because a little bit of state and system’s understanding proves sufficient most of the time

After a decade being an industry professional, I would say most is 99.5% or greater. 5 out of 1000 bugs seems about right.

So the genuinely advanced tools only start to come into play in very difficult situations - with the implication that only groups with difficult situations will _pay_ for the advanced tooling - with the implication that there's virtually no commercial market, and open source developers have to do this on their own time. Which in turn seems to be generally decreasing IMO - I believe the problem is approximately (say, 90% of the way) isomorphic to "cost of housing market" discussions essentially (how do I spend 1-2 years being someone who can understand graduate-level CS/SWE papers and write high quality code, and then write this state-of-the-art tool that no one will pay for, and still have housing, food, make debt payments etc).

Polyglot environments have to have an element of a debuggable protocol baked in at a pretty deep level to really make it work at, say, the level of Visual Studio in 1999 or something. There's all sorts of issues there around non-aligned interests that shouldn't need to exist. For example, a simple polyglot problem is "how do I send structured data from one process to another in a standardized way". The standard Unix way is "line structured", which works until it doesn't, and then you're staring at writing your own marshal/unmarshal system, which is, definitionally, only your standard. If we don't have _that_ solved, then the social pressure to have a interop debugger between, e.g., Python and Ruby is ... well, take your own guess there.

I think there's absolutely space for a hardware-software codesign to occur which builds out a hardware system, a compiler qua LLVM, a standardized interpreter layer, interop protocols, etc, and to produce a developer system that is a quantum leap beyond what we have today. But that would be a decade long project and have a very long term payoff.

the tl;dr then, imo, is that profound advancement in the field only directly benefits a few, and then only sometimes; so the incentive problem is the basic issue.

This is a very cool conversation, for some reason, I always found debugging tools to be fascinating bits of tech.

I do have to admit though, that my first read of your post was wrong tough, I read it as "debuggers used in embedded" like JTAG stuff, OpenOCD, etc. but you mean it as, if I read this right "debuggers embedded in the runtime/program" stuff like Python, or GDB server running on system right?

I do mean debuggers in embedded systems, but slightly above the JTAG level.

Think something like an Arduino.