This sounds similar to kernel debugging- is that what you have in mind? Windows (the OS I'm most familiar with) has a couple different styles of kernel debugging, which allows you to introspect various levels of system state:

https://msdn.microsoft.com/en-us/library/windows/hardware/ff...

https://msdn.microsoft.com/en-us/library/windows/hardware/hh...