News for hackers. Stuff that matters.
Re:CrackStation's Password Cracking Dictionary by earthrise on Monday February 18, 2013 @03:14PM (#5240350)
I actually didn't compare gzip and bzip2 before uploading, and I probably should have. I did use `gzip -9`. I'm compressing it with bzip2 -9 now to see if it gets any smaller. I'll post a reply here when I know.
Re:CrackStation's Password Cracking Dictionary by dfc on Monday February 18, 2013 @03:22PM (#5240391)
Use xz. Debian, Gnome, Archlinux, Gentoo, Fedora have all switched to xz, and for good reason. I expect xz will offer a 20-30% reduction in file size. That will add up quick.
Re:CrackStation's Password Cracking Dictionary by earthrise on Monday February 18, 2013 @03:29PM (#5240428)
Thanks, i'll run xz too!
Re:CrackStation's Password Cracking Dictionary by dfc on Monday February 18, 2013 @04:17PM (#5240718)
My download has slowed to a crawl around 90% completion. I am doing 3k a second. When I read that your list was not sorted case sensitively I wondered what difference that may make.

I did a little experiment with the american-wordlist-insane wordlist and sorting in the interim. I used msort which does a case sensitive sort[1] and sort which does not. Here are the results:

    6830085 insane.msort
    1461684 insane.sort.xz
    1376548 insane.msort.xz
It is not a big difference but it is still a little better. For future releases you may want to do a case sensitive sort.

[1] Zebra comes before aardvark

Re:CrackStation's Password Cracking Dictionary by earthrise on Monday February 18, 2013 @05:46PM (#5241292)
bzip2 saved about 200mb:

gzip: 4293 bzip2: 4097

xz is still running

Re:CrackStation's Password Cracking Dictionary by agent86 on Tuesday February 19, 2013 @12:58AM (#5242880)
For anyone interested, p7zip set for -mx9 got the file down to 2.72GB on my Mac.
Re:CrackStation's Password Cracking Dictionary by dfc on Tuesday February 19, 2013 @09:25AM (#5244391)
I don't know about p7zip but the xz man page cautions against blindly using -9.