Re:x86 API Hooking Demystified (2012)
by jchw on Tuesday July 09, 2019 @10:29PM
(#20398731)
Oh hey! This is a favorite topic of mine. I wrote myself a hooking library for a project where I didn’t want to use libc (and it was Win32 by design so I could just use the equivalent Win32 calls). The hardest part was definitely the LDE (length disassembly engine) and in the end I found a small header-only open source library that did it perfectly. The rest was very easy, especially on x86.
API hooking sometimes even works in hostile environments, like on software that tries to guard against patches and modification, simply because it can be challenging to detect, and you can do it early in (like inside a DLL entrypoint). So if you can do all of your work at API boundaries, you can get away with a whole lot, even if an app is packed with a strong VM packer.
Worth noting that for many less difficult use cases on Linux, you can use LD_PRELOAD to somewhat similar effect.
API hooking sometimes even works in hostile environments, like on software that tries to guard against patches and modification, simply because it can be challenging to detect, and you can do it early in (like inside a DLL entrypoint). So if you can do all of your work at API boundaries, you can get away with a whole lot, even if an app is packed with a strong VM packer.
Worth noting that for many less difficult use cases on Linux, you can use LD_PRELOAD to somewhat similar effect.