News for hackers. Stuff that matters.
Re:Visualizing Entropy in Binary Files by cortesi on Saturday November 08, 2014 @02:34PM (#8577912)
Author of the post here. I'm working on a browser-based interactive analysis tool based on these visualizations. It's nowhere near ready for release, but those who are interested can find a demo here:

http://binvis.io

This is not even an alpha yet - I plan to announce the initial release in the next month or so. Eventually, I would like to add the ability to upload, share and annotate files for analysis. Comments and criticisms welcome!

Re:Visualizing Entropy in Binary Files by 13 on Saturday November 08, 2014 @02:54PM (#8577961)
The level of detail and structure you can pick out with this tool is amazing.

https://i.imgur.com/lsRA1RN.png

Re:Visualizing Entropy in Binary Files by muyuu on Sunday November 09, 2014 @08:41AM (#8579875)
Removed?
Re:Visualizing Entropy in Binary Files by scentoni on Saturday November 08, 2014 @04:31PM (#8578211)
That's excellent! Would be very cool to also have frequency digraphs for the current segment.
Re:Visualizing Entropy in Binary Files by cortesi on Saturday November 08, 2014 @04:48PM (#8578246)
Frequency analysis and byte dot plots are definitely on the way. Do you have a concrete example of the digraph frequency feature you can link me to? Any other feature suggestions are welcome too.
Re:Visualizing Entropy in Binary Files by scentoni on Saturday November 08, 2014 @05:01PM (#8578282)
The link below to the Cantor Dust presentation shows 2d and 3d examples. I wonder if there are advantages of other space-filling curves, like http://en.wikipedia.org/wiki/Sierpi%C5%84ski_curve and http://en.wikipedia.org/wiki/Peano_curve
Re:Visualizing Entropy in Binary Files by cortesi on Saturday November 08, 2014 @05:15PM (#8578309)
Right - these are the frequency dot plots I first saw in Greg Conti's work (some examples here: http://sebug.net/paper/Meeting-Documents/BlackHat-USA2008/BH...). They're definitely on the way.

Regarding other space-filling curves - I'm in two minds about this. I've implemented z-order and the H-curve for binvis, but for different reasons neither of these are as useful as the Hilbert curve, so they're not exposed in the interface. My feeling now is that I should pick one locality preserving layout and one intuitive layout to keep things simple and usable. I want binvis to be a workaday tool, so I'm keeping my space-filling curve geekery to another project (https://github.com/cortesi/scurve).

Re:Visualizing Entropy in Binary Files by fenollp on Saturday November 08, 2014 @01:12PM (#8577657)
Have a look at Cantor Dust [1], a visual reverse-engineering program that is promising.

The author says in a presentation [2] that CD is based on Cortesi's work.

[1] https://sites.google.com/site/xxcantorxdustxx/visual-re

[2] https://www.youtube.com/watch?v=4bM3Gut1hIk

Here is a demo: https://media.blackhat.com/bh-us-12/Arsenal/Domas/_cantor.du...

Re:Visualizing Entropy in Binary Files by wxs on Saturday November 08, 2014 @04:40PM (#8578229)
These images remind me of Piet[1], an esoteric programming language where the programs are images. There you see the structure of the algorithm, rather than the binary, visually presented.

[1] http://www.dangermouse.net/esoteric/piet/samples.html

Re:Visualizing Entropy in Binary Files by pmoriarty on Saturday November 08, 2014 @10:23PM (#8578993)
This reminds me of a tool that came out 10 or 15 years ago, which allowed you to feed it arbitrary binary data that it could display in 3D.

I remember reading a Slashdot article on it in which one of the examples was a 3D, freely-rotating view of the Linux kernel binary.

Does anyone know what I'm talking about? I can't seem to recall the name of the project or find it.

Also related:

http://vimeo.com/110257380

http://gynvael.coldwind.pl/?id=199

https://www.youtube.com/watch?v=5f7hZBoEwV0

https://github.com/pwaller/binview

Re:Visualizing Entropy in Binary Files by seivadmas on Saturday November 08, 2014 @10:18PM (#8578982)
Perhaps a naive question but... what is this useful for?
Re:Visualizing Entropy in Binary Files by cyphunk on Sunday November 09, 2014 @06:33AM (#8579671)
reverse engineering binaries. as patterns are visualised you can also skip reverse engineering for some goals. for example general entropy levels of bytes (in relation to their siblings) may allow one to quickly pull out encryption keys from binaries or memory images.