News for hackers. Stuff that matters.
Re:Scripting with Go: A Modest Proposal by 1vuio0pswjnm7 on Sunday August 20, 2023 @06:11PM (#37203404)

      export LC_ALL=C
      awk '!a[$1]++' access.log|head 
If access.log is large enough, awk will fail.

When this happens, one can split access.log into pieces, process separately then recombine.

But that's more or less what sort(1) does with large files, creating temporary files in $TMPDIR or other user-specified directory after -T if using GNU sort.

There was a way to eliminate duplicate lines from an unordered list using k/q, without using temporary files but I stopped using it after Kx, Inc. was sold off and I started using musl exclusively. q requires glibc.

For example, something like

     #!/bin/sh
     # usage: $0 file
     echo "k).Q.fs[l:0::\`:$1];l:?:l;\`:$1 0:l"|exec q >null;
Can this be done in ngn k.

The other approach I use to avoid temporary files is to just put the list in an SQL database, add a UNIQUE constraint, and update the database.

Re:Scripting with Go: A Modest Proposal by ramranch on Sunday August 20, 2023 @07:39PM (#37203987)
Out of curiosity, what's your reasoning for using musl exclusively
Re:Scripting with Go: A Modest Proposal by 1vuio0pswjnm7 on Sunday August 20, 2023 @08:04PM (#37204129)
musl comes as an option on the Linux distribution I'm using.

I do a lot of static compilation.

I generally prefer smaller, simpler stuff.

I only started using Linux recently. Until then I used NetBSD, and their libc is much nicer I think. Static compilation is of course no problem on NetBSD.

NB. These are four points that apply to me, but not necessarily to anyone else, i.e., 1. the distribution I use, 2. what I like to do, i.e., compile static binaries, 3. particular personal preferences, i.e., minimalism, limitation that forces creativity, and 4. I did not switch from glibc to musl, I switched from NetBSD libc to musl.

It's also fun when stuff occasionally does not compile in that it forces me to learn about certain differences between musl and glibc and usually I learn something I did not know about C or the compilers.

I'm not the only person in the world who prefers musl. For example,

https://raw.githubusercontent.com/hardbol/spitbol/master/rea...