News for hackers. Stuff that matters.
Re:A Sampling of Anti-Decompilation Techniques by glandium on Thursday November 16, 2017 @08:50PM (#15718353)
I don't know if decompilers are able to do something about it, but there's a "neat" technique where your machine code can be interpreted as two different sequences of instructions depending where you start instruction decoding. For a simple but artificial example of what I mean:

The following sequence of bytes:

  b8 50 83 ec 10
Decodes as:

  mov $0x10ec8350, %eax
if you start at the first byte, and, if you start at the second byte, as:

  push %eax
  sub $0x10, %esp
Here I essentially hid an instruction in the mov'ed data, because that's the easiest way to create something like that, but I've seen mind blowing examples of this technique. I unfortunately don't remember where.
Re:A Sampling of Anti-Decompilation Techniques by psykotic on Thursday November 16, 2017 @09:54PM (#15718559)
That's an issue for static disassembly, which is step one of decompilation. Your snippet is a classic example that can throw off naive linear-scan disassemblers. Recursive disassemblers can handle it easily if (and it's a big if) they can identify the basic block entry points. If you have top-level entry points (which can also be a problem), the main problem for a recursive disassembler are indirect jumps that don't fit standard patterns like switch jump tables.

All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points.

Re:A Sampling of Anti-Decompilation Techniques by munin on Thursday November 16, 2017 @10:24PM (#15718650)
> All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points.

That's only true if the traces you can generate do a reasonable job at covering the states the application can find itself in, which is a big assumption.

Re:A Sampling of Anti-Decompilation Techniques by psykotic on Friday November 17, 2017 @01:19AM (#15719484)
Yes, that's what I meant by coverage of the relevant branches. But the good news is that all these partial, heuristic sources of information can be combined. E.g. for detecting function entry points, you can combine information from ELF/PE export tables (if present), function prologues detected by a linear scan, vtables, static CALL targets, dynamic CALL targets from run-time traces, etc.
Re:A Sampling of Anti-Decompilation Techniques by reacweb on Friday November 17, 2017 @04:14AM (#15720421)
It is used in windows 3.1 and explained in "undocument windows" https://www.amazon.com/Undocumented-Windows-Programmers-Micr...