News for hackers. Stuff that matters.

A comment on Batch editing files with ed.

Re:Batch editing files with ed by davidgould on Saturday May 12, 2018 @05:33PM (#17056125)
Awk really is the tool of choice for this sort of thing:

  $ awk '{print}/baz/ {sub("baz", "elephant"); print}' jvns.txt
  foo:
    - bar
    - baz
    - elephant
    - bananas
Since the script is single quoted you can also lay it out legibly:

  $awk '
  {print}
  /baz/ {
      sub ("baz", "elephant")
      print
  }
  '
which is nice for more complex "one liners". Awk is also standard on all posix environments and in the mawk flavor is extremely fast (relevant if you are processing huge files).

There is superb book, The AWK Programming Language, which teaches a lot about programming in addition to the awk language. Good discussion and link to pdf here: [0]

[0] https://news.ycombinator.com/item?id=13451454

[ Parent | on HN ]
Re:Batch editing files with ed by lucidguppy on Saturday May 12, 2018 @06:58PM (#17056463)
The AWK Programming Language is the best programming book ever because it lets you learn the language through interesting problems (like writing a very small assembler).
[ Parent | on HN ]
Re:Batch editing files with ed by lemonberry on Saturday May 12, 2018 @07:51PM (#17056645)
It's available online. Thanks for the tip.

https://ia802309.us.archive.org/25/items/pdfy-MgN0H1joIoDVoI...

[ Parent | on HN ]
Re:Batch editing files with ed by kaushalmodi on Sunday May 13, 2018 @09:03AM (#17058693)
Come on, don't promote piracy of the book! It's worth buying! Mods, please take down this link.
[ Parent | on HN ]
Re:Batch editing files with ed by kaushalmodi on Sunday May 13, 2018 @12:59PM (#17059835)
Hmm, someone downvoted me. Can the downvoter explain what was wrong in what I said?
[ Parent | on HN ]
Re:Batch editing files with ed by lemonberry on Sunday May 13, 2018 @04:26PM (#17060872)
I believe the book's been out of print for sometime. I've no idea if that's why you were downvoted. I wouldn't post a link to a pirated anything. That pdf is linked from a ton of sites and the site I found it on seemed like a reputable site, though I don't remember which it was at the moment.
[ Parent | on HN ]
Re:Batch editing files with ed by kaushalmodi on Sunday May 13, 2018 @08:15PM (#17061799)
> I believe the book's been out of print for sometime.

Hmm, I did not know that (I have a print copy of that book).

> I wouldn't post a link to a pirated anything.

OK. But just in case, someone still wants a physical book, they can get it used from Amazon (can talk about Amazon in US) for ~$3.

[ Parent | on HN ]
Re:Batch editing files with ed by mjn on Sunday May 13, 2018 @07:45AM (#17058449)
> the mawk flavor is extremely fast

Fast, but partly because it's not Unicode-aware: it treats strings as 8-bit character sequences rather than UTF-8. Often that's fine, if non-ASCII characters are only passed through unmodified, but requires some care to avoid problems.

    $ echo $LANG
    en_GB.UTF-8

    $ echo "ÜNICÖDE" | gawk '{print tolower($0)}'
    ünicöde

    $ echo "ÜNICÖDE" | mawk '{print tolower($0)}'
    �nic�de
I ran into this in practice because I was using awk to convert paper titles from "Title Case" to APA-style "Only first word of title capitalized" case. The garbled output led me down a rabbit hole where I discovered that only some awks support Unicode locales, and the default awk on Debian (mawk) isn't one of them.
[ Parent | on HN ]
Re:Batch editing files with ed by davidgould on Sunday May 13, 2018 @09:08AM (#17058721)
See my post way down the bottom, I do mention this. GNU awk also has a lot of useful extensions and builtins so that sometimes it's painful to use a plain Posix awk. But, when you need the speed, it's nice to know mawk is out there.

I'd love to see the mawk compilation technology merged to GNU awk. Or mawk updated with Unicode support and a few of the GNU extensions.

The other item on my awk-like wishlist is CSV support, ie split $1 .. $N by CSV rules instead of just a field separator. I usually end up copying CSVs into postgresql because it is fast and then I can process it very flexibly, but it's a bit heavy for things that could be one line-ers. Also, postgresql won't load malformed CSV, but I suspect awk could be less picky.

There is miller which hits some of these points, but I haven't really grokked it yet and the syntax seems ... awkward.

[ Parent | on HN ]
Re:Batch editing files with ed by kaushalmodi on Sunday May 13, 2018 @08:55AM (#17058662)
+1 for The AWK Programming language book!

I have been reading through it and running code snippets from there as I find time. It's awesome!

If anyone's interested, I also update my notes from there on my blog.. https://scripter.co/notes/awk. I plan to have that notes post contain the entire book when I am done.

[ Parent | on HN ]