News for hackers. Stuff that matters.
Re:Awk As A Major Systems Programming Language, Revisited (2... by jumpman500 on Wednesday February 12, 2020 @12:40AM (#22306131)
Awk and sed are generally superior for simple data transformations than python. Like cutting off specific lines, columns or replacing characters in files. They have pretty great performance in large files.

Once things get complicated though you should probably push whatever transformations you need to do to a database.

Re:Awk As A Major Systems Programming Language, Revisited (2... by manifoldgeo on Thursday February 13, 2020 @12:06AM (#22315454)
I would like to offer a counter-argument. I’ve been torrenting TV series for years, and I always have to do a mass rename of The files afterward to get Plex to process them properly and download the metadata. They’re always named something like “TV Show Name 2014.L337.H4XXX1080p-420.mkv”.

The other day I set out to use BASH to mass rename, and I had such trouble backslash-escaping the dot characters and whitespaces to prevent sed from interpreting them specially. Eventually I gave up and searched for the pythonic way to rename them, and it was as simple as a string replacement followed by a call to os.rename() inside a for-loop. It was a breath of fresh air to escape “command line Kung Fu” and fearing the thrashing from shell globbing.

To be fair, I got my start in using sed and awk as powertools in the BASH command pipeline, but I don’t miss them compared to a language with strong data types and simple built-in methods for handling complex manipulation. Python is built in to basically every Linux distro that has BASH, and for the sake of simple transformations, it offers a lot of succinct methods that work on either 2.7 or 3.x with no external packages.

Re:Awk As A Major Systems Programming Language, Revisited (2... by jumpman500 on Thursday February 13, 2020 @11:51PM (#22324029)
Your example shows a very particular shortcoming in the unix command line programs. The mv command doesn't play nice with spaces in file names. I think python is a fine solution for your use case and wouldn't recommend against your code. Especially if you had to build it out for a lot of different patterns.

But if you're dealing with changing stuff in text files greater than 25 gigs I'd always recommend using the unix cmd line tool set for simple modifications if performance is a key concern. It works the same everywhere and can rip through big files. But I'm a Vim guy and I live in the shell so I'm surely quite biased lol.

There are plenty of pretty easy one liners though that can rename files depending on if you can define a pattern correctly though.

echo 'TV Show Name 2014.L337.H4XXX1080p-420.mkv' | awk -F . -v q="'" '{print q$0q" "q$1".mkv"q }' | xargs mv

find . -type f -name '*.mkv' -printf '%f' | awk -F . -v q="'" '{print q$0q" "q$1".mkv"q}' | xargs mv

Re:Awk As A Major Systems Programming Language, Revisited (2... by beojan on Thursday February 13, 2020 @02:31AM (#22315990)
You need zmv
Re:Awk As A Major Systems Programming Language, Revisited (2... by manifoldgeo on Thursday February 13, 2020 @07:49PM (#22322929)
Thanks! I hadn't heard of zmv before, and it fits my use case perfectly.
Re:Awk As A Major Systems Programming Language, Revisited (2... by markus_zhang on Wednesday February 12, 2020 @01:32PM (#22311348)
Thanks this is a good point. I did read some posts here that use sqlite for sort of middle storage in the pipelines.