News for hackers. Stuff that matters.
Re:Avast open-sources its machine-code decompiler by ggambetta on Wednesday December 13, 2017 @09:40AM (#15914456)
Is there any way to use this on a DOS EXE? This would be a lovely tool to port/remaster old games (i.e. decompile, replace rendering methods with modern equivalents, compile again -- an "offline" version of the "online" idea I was going for with http://www.gabrielgambetta.com/remakes.html)
Re:Avast open-sources its machine-code decompiler by partycoder on Wednesday December 13, 2017 @02:49PM (#15916993)
Note that it was not rare for some old DOS games to have bits of assembly in them.

This was usually done to overcome some performance bottleneck but with today's hardware you might not need that at all.

Re:Avast open-sources its machine-code decompiler by khedoros1 on Wednesday December 13, 2017 @05:06PM (#15918111)
Yesteryear's C compilers weren't great, optimization-wise. But anything remotely modern would just brute force its way through a DOS-era program, well optimized or not.
Re:Avast open-sources its machine-code decompiler by partycoder on Wednesday December 13, 2017 @05:16PM (#15918197)
Also yesterday's instruction sets were a bit more limited.
Re:Avast open-sources its machine-code decompiler by khedoros1 on Wednesday December 13, 2017 @04:37PM (#15917908)
It probably needs a profile for the MZ format. And probably for 16-bit x86. I'm considering trying it on some .com binaries, to see what it does (since it said it can handle raw binaries). I doubt it'll work immediately, because I doubt that their code handles memory segmentation properly, 16 bit pointers and data, etc.
Re:Avast open-sources its machine-code decompiler by israrkhan on Wednesday December 13, 2017 @10:42AM (#15914907)
Cool. Finally some free alternative to IDA decompiler plugin. IDA is still better due to interactive nature, i.e. you can explore the code and rename variables/functions as you keep on exploring. I hope this evolves into something like that.
Re:Avast open-sources its machine-code decompiler by milcron on Wednesday December 13, 2017 @02:54PM (#15917046)
You might be interested in the free decompiler Radare2 https://en.wikipedia.org/wiki/Radare2
Re:Avast open-sources its machine-code decompiler by chainsaw10 on Thursday December 14, 2017 @01:16AM (#15920309)
radare2 is a disassembler, not a decompiler.

Disassembly is a much easier task than decompilation, since it's a mostly mechanical process. Decompilation requires you to undo the optimizations/transformations the compiler did as it generated the binary, which is much harder.

That said radare2 is still cool, and a GUI (Cutter) is in the works.

Re:Avast open-sources its machine-code decompiler by ZeroCool2u on Wednesday December 13, 2017 @12:08PM (#15915549)
Exactly what I was thinking. I remember my sophomore year of CS undergrad I emailed IDA's support asking if there was a student license or some equivalent to learn how to use it. Props to them for at least replying, but their answer was a firm no and was disappointing to say the least.
Re:Avast open-sources its machine-code decompiler by phire on Wednesday December 13, 2017 @12:53PM (#15915925)
When I was a student, I once tried to buy a copy of IDA Pro Advanced, as the Standard version didn't have the features I needed.

They refused to sell it to me, unless I bought the standard version and used that for a year (which in retrospect, I perhaps should have done)

Re:Avast open-sources its machine-code decompiler by lbenes on Wednesday December 13, 2017 @05:06PM (#15918106)
It's crazy that companies don't fully recognizes this. Any lost revenue to students would more than made up for when they join the workforce.
Re:Avast open-sources its machine-code decompiler by pc2g4d on Wednesday December 13, 2017 @04:36PM (#15917900)
Backward R in the logo: this drives me nuts. To any readers of the Cyrillic alphabet it says "Ya". So this says "Yaetargetable Decompiler"

I can't be the only one!

Re:Avast open-sources its machine-code decompiler by praptak on Wednesday December 13, 2017 @04:48PM (#15917975)
I'm not even Russian and I find this "funny" usage of "Я" and sometimes "И" rather grating.
Re:Avast open-sources its machine-code decompiler by Dylan16807 on Friday December 15, 2017 @06:14AM (#15931028)
It's not written in a Cyrillic language. It's written in English.

Nobody comes along and points out that in Spanish the "g" works differently, and it bugs them to see words with "g" in them.

It's one thing to do faux-Cyrillic and get the letters wrong. It's quite another to do something silly to a latin letter, and get complaints that it resembles a non-latin letter.

Re:Avast open-sources its machine-code decompiler by pc2g4d on Friday December 15, 2017 @02:39PM (#15934401)
It doesn't resemble the Cyrillic letter---it is it. "R" is an English letter but not a Cyrillic one, and "Я" is a Cyrillic letter but not an English one, and by flipping them horizontally you transform them into each other. I imagine for many of the 7,574,303 people in Russia who speak English and probably also a fair chunk of the 854,955 Americans who speak Russian (and presumably have mastered both alphabets), it's annoying. Not a huge deal, just annoying.

https://en.wikipedia.org/wiki/List_of_countries_by_English-s...

https://en.wikipedia.org/wiki/Russian_language_in_the_United...

Re:Avast open-sources its machine-code decompiler by Dylan16807 on Friday December 15, 2017 @05:02PM (#15935606)
There are so many symbols from different languages that resemble each other. If I use a smiley face, that doesn't mean I used a "ü" or a "ツ" from another alphabet just because it looks similar. A backwards R is visually the same as a Cyrillic character, but that doesn't mean I'm writing in Cyrillic, just like a "P" is visually the same as a Cyrillic character but doesn't mean I'm writing in Cyrillic.
Re:Avast open-sources its machine-code decompiler by skate22 on Wednesday December 13, 2017 @06:43AM (#15913624)
Does anyone know why intel discontinued their tamper protection toolkit? They had an obfuscation compiler that would turn compiled C code into a self encrypting/decrypting code. The idea was if you dissassembled the code at any point you wpuld get mostly garbage instructions. I always wondered how a de compiler could get around that.

https://software.intel.com/en-us/articles/intel-tamper-prote...

Re:Avast open-sources its machine-code decompiler by _wmd on Wednesday December 13, 2017 @08:04AM (#15913942)
Google for anything Rolf Rolles has published on the topic, believe it or not there are general approaches to solving this. Someone already mentioned dumping the text segment, that only works for silly 90s-era obfuscators.

Contemporary obfuscators _rewrite_ the protected code as a series of instructions executed on a virtual machine whose bytecode (and bytecode semantics!) are randomly generated at build time. The solution (AIUI) is symbolic execution of the instructions to determine their underlying architectural effect, synthesize some compiler IR that is equivalent to those effects, run an optimization pass (like a regular compiler) over that IR, and finally generate x86 from the result.

The optimization passes are necessary to remove side effects that do not impact the state of the program ("noise"), which modern obfuscators like Themida insert a ton of into the instruction stream

In other words, rather than attempt to dump some particular part of the program, the binary as a whole is statically analysed to determine, regardless of the indirections inserted by any obfuscation pass, what machine instructions are ultimately executed for a given program input. The abstract representation is then compiled to an equivalent new program which is much easier to read, because all of the indirections and noise have been optimized away.

When I was reading about Rolles' work initially, I couldn't help but imagine this is the kind of approach Geordi La Forge would have come up with if cracking an encrypted binary were ever the plot for an episode of Star Trek :)

Re:Avast open-sources its machine-code decompiler by j_s on Wednesday December 13, 2017 @11:17AM (#15915150)
instructions executed on a virtual machine whose bytecode (and bytecode semantics!) are randomly generated at build time

Like the one built into Windows: https://github.com/airbus-seclab/warbirdvm

Re:Avast open-sources its machine-code decompiler by nekitamo on Wednesday December 13, 2017 @09:23AM (#15914352)
iirc it wasn't very advanced. They would 'mov' the decrypted instructions to a region in memory (always the same one), executed it, then save register state and go on to decrypt the next set of instructions.

Breaking it involved monitoring the memory for the decrypted instructions, and dumping them right before they were executed. I don't remember if there were any additional complications with stuff like conditional jumps.

Re:Avast open-sources its machine-code decompiler by heavenlyblue on Wednesday December 13, 2017 @07:25AM (#15913771)
This is something that code protectors and viruses have been doing for ages. Not surprised Intel couldn't make this into a sellable product.
Re:Avast open-sources its machine-code decompiler by alfiedotwtf on Wednesday December 13, 2017 @07:27AM (#15913781)
How would this work if you were to run it under QEMU and just dump the code segment after decryption?
Re:Avast open-sources its machine-code decompiler by PeterisP on Wednesday December 13, 2017 @07:33AM (#15913810)
The point is that there is no "after decryption", at any given moment in time only a small portion of the code is decrypted.
Re:Avast open-sources its machine-code decompiler by akrasuski1 on Wednesday December 13, 2017 @07:47AM (#15913864)
There has been some malware that did just that - it was still possible to record the trace of instructions being executed along with the current instruction pointer to be able to reconstruct the binary quite well.
Re:Avast open-sources its machine-code decompiler by bogomipz on Wednesday December 13, 2017 @11:07AM (#15915088)
>"As we announced in our Botconf 2017 presentation at the beginning of December (slides), RetDec, our machine-code decompiler, is now open, which means anyone can freely use it, study its source code, modify it, and redistribute it."

These slides linked in the above looks like this was a really fascinating talk.

Does anybody know when or if this presentation was recorded or if it will be made available? I would love to watch this.

Re:Avast open-sources its machine-code decompiler by j_s on Wednesday December 13, 2017 @11:14AM (#15915128)
There was a live stream, but nothing appears to be available yet.

https://twitter.com/Botconf

Re:Avast open-sources its machine-code decompiler by bogomipz on Wednesday December 13, 2017 @11:59AM (#15915467)
Yeah I also checked youtube with various search terms and came up empty. Hopefully someone from Avast or Botconf will read this and post them :)

It looks like some conference presentations from years past have made it to youtube.

Re:Avast open-sources its machine-code decompiler by Direct on Wednesday December 13, 2017 @06:36AM (#15913605)
Really cool stuff. I don't like being negative when it comes to fantastic moves like this, but I'm still really disappointed that it doesn't support 64bit executables.
Re:Avast open-sources its machine-code decompiler by SkyPuncher on Wednesday December 13, 2017 @09:35AM (#15914432)
A reddit comment had a pretty good explanation.

Most malicious code is still written in 32-bit since 64-bit Windows supports running 32-bit code.

Write something in 32-bit - target 100% of devices. Write something in 64-bit - target ~50% of devices.

Re:Avast open-sources its machine-code decompiler by Zuider on Wednesday December 13, 2017 @05:59PM (#15918503)
But if security software only treats 32-bit executables as suspicious, wouldn't it make sense for malware creators to switch to 64-bit?
Re:Avast open-sources its machine-code decompiler by firethief on Wednesday December 13, 2017 @09:33PM (#15919539)
Is any security software that easily bypassed?
Re:Avast open-sources its machine-code decompiler by monocasa on Wednesday December 13, 2017 @11:11AM (#15915110)
Reddit implies that it's on their roadmap.

https://www.reddit.com/r/programming/comments/7jhk6p/avast_o...

Re:Avast open-sources its machine-code decompiler by chrisseaton on Wednesday December 13, 2017 @08:02AM (#15913933)
Maybe that's why they're open sourcing this one? They have their own internal new one that does support AMD64.
Re:Avast open-sources its machine-code decompiler by MaxBarraclough on Wednesday December 13, 2017 @07:16AM (#15913735)
Agreed.

x86, ARM, MIPS, PIC32, PowerPC, but not x86-64. Impressive list, but an odd choice.

Re:Avast open-sources its machine-code decompiler by jjoonathan on Wednesday December 13, 2017 @09:04AM (#15914227)
IDA gives away x86 and charges for x86-64, maybe they're going for a similar freemium model?
Re:Avast open-sources its machine-code decompiler by luch on Wednesday December 13, 2017 @10:53AM (#15914993)
I think the problem is technical rather than commercial : https://github.com/avast-tl/retdec/issues/9.

x86_64 have calling conventions (namingly __fastcall) which are more inconvenient to decode than x86 _cdecl or __stdcall where every arguments are passed on the stack. Most symbolic engines usually works only on x86 for the same reason.

Re:Avast open-sources its machine-code decompiler by taspeotis on Wednesday December 13, 2017 @02:52AM (#15912739)
I've used retdec before, its output is quite nice. I even had some problems with it (doing dumb stuff like putting in executables that were beyond the limits they imposed on their website) and whoever they had supporting it were quite friendly in helping me anyway.
Re:Avast open-sources its machine-code decompiler by Tobba_ on Wednesday December 13, 2017 @07:12AM (#15913712)
Looks like it's also relying on LLVM for disassembly? Ouch; that's an incredibly bad idea if you're trying to analyze malicious or unusual code (it's not designed for that), but I guess it's the easiest for a proof of concept like this.

Although, there's no way an AV company doesn't have its own disassembler, but those are almost always treated as trade secrets (especially the stuff that isn't in the spec / the spec is wrong). They'll probably hook it up to that before doing any real work with it themselves.

Re:Avast open-sources its machine-code decompiler by bringtheaction on Wednesday December 13, 2017 @07:49AM (#15913876)
> proof of concept

They've been working on this for 7 years they said so I don't think it counts as just a PoC.

Re:Avast open-sources its machine-code decompiler by UncleEntity on Wednesday December 13, 2017 @07:47AM (#15913867)
> Looks like it's also relying on LLVM for disassembly?

[wild speculation here] I suspect they're using llvm to go from an ast to c(++) code since they have tooling for stuff like that.

Now I have to find me a binary-blob kernel module that manufactures like to put out and see what the C code it spits out looks like -- another wasted day methinks...

Re:Avast open-sources its machine-code decompiler by Tobba_ on Wednesday December 13, 2017 @02:34PM (#15916848)
You could just ..uh.. acquire a copy of IDA that has the AMD64 decompiler. It's more mature and spits out C code of wildly varying readability, though only for one function at a time.
Re:Avast open-sources its machine-code decompiler by Karliss on Wednesday December 13, 2017 @01:26PM (#15916220)
Isn't it using capstone?
Re:Avast open-sources its machine-code decompiler by Tobba_ on Wednesday December 13, 2017 @02:31PM (#15916815)
Doesn't look that way? https://github.com/avast-tl/retdec/blob/master/src/bin2llvmi...

Capstone would probably be the best open-source choice for something like this though.

Re:Avast open-sources its machine-code decompiler by Karliss on Wednesday December 13, 2017 @05:12PM (#15918161)
Re:Avast open-sources its machine-code decompiler by tonetheman on Wednesday December 13, 2017 @07:13AM (#15913720)
Super cool move. Always interesting to look at techniques for this type of stuff. It always feels like black magic.
Re:Avast open-sources its machine-code decompiler by maxton on Wednesday December 13, 2017 @10:58AM (#15915028)
This is awesome! I had been using retdec.com since before Avast bought AVG (where RetDec was originally developed). I'm very excited to do away with the limits the website imposed.
Re:Avast open-sources its machine-code decompiler by gaspoda on Wednesday December 13, 2017 @06:42AM (#15913622)
Why they do it? This is their competitive advantage.
Re:Avast open-sources its machine-code decompiler by heavenlyblue on Wednesday December 13, 2017 @07:26AM (#15913779)
Modern AV software uses a virtual machine and a decompiler in tandem. This is not their only competitive advantage.
Re:Avast open-sources its machine-code decompiler by drej on Wednesday December 13, 2017 @06:54AM (#15913662)
Now the usual for large projects like this one (this is just for the main repo):

  $ loc .
  --------------------------------------------------------------------------------
   Language             Files        Lines        Blank      Comment         Code
  --------------------------------------------------------------------------------
   C++                    587       202592        23441        43727       135424
   C/C++ Header           450        34934         6371        11733        16830
   Bourne Shell            10         2363          247          518         1598
   Plain Text              16          827           46            0          781
   Autoconf                 1         2507          551         1635          321
   Python                   1          195           32           22          141
   Markdown                 2          162           45            0          117
   ASP.NET                  2            2            0            0            2
  --------------------------------------------------------------------------------
   Total                 1069       243582        30733        57635       155214
  --------------------------------------------------------------------------------
Re:Avast open-sources its machine-code decompiler by jwilk on Wednesday December 13, 2017 @07:14AM (#15913722)
> Now the usual for large projects like this one (this is just for the main repo):

I don't understand what you're trying to say. The usual... what?

Re:Avast open-sources its machine-code decompiler by drej on Wednesday December 13, 2017 @07:23AM (#15913763)
Oh, I thought that we do this sort of thing whenever there's a new open source project posted here. People seem to want to know what it's made of and potentially its (S)LOC.
Re:Avast open-sources its machine-code decompiler by chrisseaton on Wednesday December 13, 2017 @07:41AM (#15913843)
I've never seen anyone do that on here.
Re:Avast open-sources its machine-code decompiler by exikyut on Wednesday December 13, 2017 @09:05AM (#15914233)
But I like it. GitHub is all like "98.6% C++, 1.4% Other", which is... an understandable heuristic, but very meh.
Re:Avast open-sources its machine-code decompiler by chrisseaton on Wednesday December 13, 2017 @09:21AM (#15914336)
I wish GitHub let me customise which directories to include. My project shows up as 74% Ruby, but a lot of that is some specs that aren't really part of the project, and in reality it's 60% Ruby.
Re:Avast open-sources its machine-code decompiler by jwilk on Wednesday December 13, 2017 @12:32PM (#15915741)
To some extent, you can influence GitHub's classification via gitattributes:

https://github.com/github/linguist

Re:Avast open-sources its machine-code decompiler by Fnoord on Wednesday December 20, 2017 @12:35PM (#15971751)
Didn't know about loc or this thing, thanks for bringing it up.
Re:Avast open-sources its machine-code decompiler by oneweekwonder on Wednesday December 13, 2017 @10:12AM (#15914685)
I wonder if there is a website/api that will give you loc of pypi or npm packages.

Maybe in the form of a forum badge or certificate like pdf.

Re:Avast open-sources its machine-code decompiler by slowmotiony on Wednesday December 13, 2017 @07:36AM (#15913822)
So there are two lines of ASP.NET code? Do I read this correctly?
Re:Avast open-sources its machine-code decompiler by dspillett on Wednesday December 13, 2017 @08:04AM (#15913940)
Two single line files that have a file extension making them look like ASP.net code is how I read that.

It could be place-holder scripts that redirect to the correct location, for instance a default.asp/index.asp file that does nothing but redirect to index.html some other default that IIS doesn't recognised out of the box. This would catch cases where someone has just dumped the web assets on a IIS share (IIS doesn't, or at least didn't used to, consider index.html as a potential default document). In classic ASP this would be something like the line:

    <% response.redirect "index.whatever-extension" %>
It could also be false positives in whatever test is being performed. For a project like this, I suspect that is more likely to be the case.
Re:Avast open-sources its machine-code decompiler by bringtheaction on Wednesday December 13, 2017 @07:48AM (#15913873)
Probably a misclassification. It's not 100% accurate but it gives a good idea of the size and languages of the code base.