There are many AWK implementations that are obscenely portable. A case in point is busybox AWK. Another might be http://unxutils.sourceforge.net

A complex AWK script is very, very easy to move to a platform that lacks any AWK parsers. It can easily be done on Windows, without administrative rights, by the placement of a single .EXE - Perl can do many things, but that is not one of them (to the best of my knowledge).

perl borrowed stuff from awk, so you could also do

    perl -ne 'print if !$seen{$_}++'

Play a round of perl golf?

    perl -pne '$_=$#$_++?$_:""'

I'm rusty at this but shaved off six chars, five if you count the 'p' added to switches.

I have never seen this $#$var trick and google is not the friend of perl operators. Do you have any explanation ?

    perl -ne 'print if!++$#$_'

seems to work also

If you have array @foo in perl, $#foo is the index of the last element of the array, which is just the size of the array -1. So if @foo is undefined, $#foo is -1.

Using a variable instead of 'foo' is a symbolic reference, so this is effectively using the symbol table as the associative array. This means that this solution also gets it wrong if your file contains a line that matches the name of a built-in variable in perl. That would be tough to debug!

If your file contains

    This is the first line of the file

then during execution of

    ++$#$_

the result is the same as if you had written

    ++$#{This is the first line of the file}

So the variable @{This is the first line of the file} goes from undefined to an array of length 1, turning $#{This is the first line of the file} to 0.

Incidentally, this is why the snippet fails to work for a line repeated more than once: for each occurrence of the expression, the value returned is in the sequence -1, 0, 1, 2, 3, ... so it is only false for the second occurrence.

Using preincrement instead of postincrement means the values returned are 0, 1, 2, 3, ... which means that inverting the test makes it false for every occurrence after the first.

edit: this is failing if a line is repeated more than once, see @showdead's excellent explanation

the shortest I've got so far is

    perl -lnE'say if!++$#$_'

---

I don't understand what's happening with $#$_ but seems like something I should look into, thanks :)

you could remove n as p is used and would be same no. of characters as

    perl -ne 'print if $#$_++'

you could save one more by removing space between e switch and single quote