News for hackers. Stuff that matters.
Re:Removing duplicate lines from files keeping the original ... by asicsp on Wednesday May 29, 2019 @05:21AM (#20038179)
perl borrowed stuff from awk, so you could also do

    perl -ne 'print if !$seen{$_}++'
Re:Removing duplicate lines from files keeping the original ... by rlonstein on Wednesday May 29, 2019 @09:00AM (#20039583)
Play a round of perl golf?

    perl -pne '$_=$#$_++?$_:""'
I'm rusty at this but shaved off six chars, five if you count the 'p' added to switches.
Re:Removing duplicate lines from files keeping the original ... by reacweb on Wednesday May 29, 2019 @10:34AM (#20040661)
I have never seen this $#$var trick and google is not the friend of perl operators. Do you have any explanation ?

    perl -ne 'print if!++$#$_'
seems to work also
Re:Removing duplicate lines from files keeping the original ... by showdead on Wednesday May 29, 2019 @10:53AM (#20040872)
If you have array @foo in perl, $#foo is the index of the last element of the array, which is just the size of the array -1. So if @foo is undefined, $#foo is -1.

Using a variable instead of 'foo' is a symbolic reference, so this is effectively using the symbol table as the associative array. This means that this solution also gets it wrong if your file contains a line that matches the name of a built-in variable in perl. That would be tough to debug!

If your file contains

    This is the first line of the file
then during execution of

    ++$#$_
the result is the same as if you had written

    ++$#{This is the first line of the file}
So the variable @{This is the first line of the file} goes from undefined to an array of length 1, turning $#{This is the first line of the file} to 0.

Incidentally, this is why the snippet fails to work for a line repeated more than once: for each occurrence of the expression, the value returned is in the sequence -1, 0, 1, 2, 3, ... so it is only false for the second occurrence.

Using preincrement instead of postincrement means the values returned are 0, 1, 2, 3, ... which means that inverting the test makes it false for every occurrence after the first.

Re:Removing duplicate lines from files keeping the original ... by asicsp on Wednesday May 29, 2019 @10:29AM (#20040580)
edit: this is failing if a line is repeated more than once, see @showdead's excellent explanation

the shortest I've got so far is

    perl -lnE'say if!++$#$_'
---

I don't understand what's happening with $#$_ but seems like something I should look into, thanks :)

you could remove n as p is used and would be same no. of characters as

    perl -ne 'print if $#$_++'
you could save one more by removing space between e switch and single quote