News for hackers. Stuff that matters.
Re:Ask HN: Why isn’t visual programming a bigger thing? by JoshuaDavid on Thursday May 21, 2020 @09:48PM (#23267901)
Yes, it's not particularly difficult either, as long as you have the appropriate tools installed.

    csvjson path/to/infile.csv \
        | jq -c ".[]" \
        | while read LINE; do
            UNAME="$(echo "$LINE" | jq -r ".uname")";
            CNAME="$(echo "$LINE" | jq -r ".cname")";
            if id -u "$UNAME" &> /dev/null; then
                # Find processes with this command name belonging to this user
                comm -12 <(ps -C "$CNAME" -o "pid=" | sort) <(ps -U "$UNAME" -o "pid=" | sort) | \
                    while read PID; do
                        # rss=Resident Set Size, i.e. the amount of memory this process is currently
                        # using, not including swap. You might want vsize instead, depending on your needs.
                        RSS="$(ps -o "rss=" "$PID")";
                        STARTISODATE="$(date --date="$(ps -o "lstart=" "$PID")" --iso-8601="seconds")";
                        # If you actually plan to use this you need to make sure pid isn't anything important
                        kill "$PID";
                        jq -n -c \
                            --argjson "pid" "$PID" \
                            --argjson "rss" "$RSS" \
                            --arg "start" "$STARTISODATE" \
                            --arg "command" "$CNAME" \
                            --arg "username" "$UNAME" \
                            '{"pid":$pid,"memory_kb":$rss,"start":$start,"command":$command,"username":$username}';
                    done
            else
                # Print an error to stdout if the user does not exist
                echo "warning: user $UNAME does not exist" >&2
            fi
        done \
        | jq -s 'sort_by(.memory_kb)' \ # sort_by(-.memory_kb) if you want it in descending order
        | in2csv --format=json \
        > path/to/outfile.csv

I'm guessing you meant "parse the CSV correctly using only bash", and I agree that's difficult, which is why you should not do that. Instead, you should use a tool that has been specifically designed to do that one task, and do it well (in this case, that task is "convert a csv into a format that's easier to work with, namely json"). This is the unix way.

How extensible is PowerShell? With bash, you can write a new command that takes text input in a certain format and produces text output in a certain format, in whatever language you want, and plug it in in the appropriate place in the pipeline. Is the same true of PowerShell?

Re:Ask HN: Why isn’t visual programming a bigger thing? by jiggawatts on Friday May 22, 2020 @05:49AM (#23270291)
I am properly impressed. Wow!

Most people I give this challenge to fail to account for the complexities of even a simple format like CSV, and typically simply use "grep" to find processes, which would cause havoc with simple user names like "Bob Ash = bash".

For comparison, the equivalent in PowerShell is this:

    #require -RunAsAdministrator
    $i = ConvertFrom-Csv @'
    UserName,ProcessName
    DOMAIN\BobJones,explorer
    DOMAIN\Bash,notepad
    '@
    
    $ps = Get-Process -IncludeUserName
    Compare-Object $ps $i -PassThru -Property UserName,ProcessName -ExcludeDifferent -IncludeEqual  `
        | Stop-Process -PassThru `
        | Sort-Object PeakWorkingSet64 -Descending `
        | Select-Object `
            ProcessName, `
            PeakWorkingSet64,`
            @{ Name='StartDate'; Expression={ '{0:o}' -f $_.StartTime } } `
        | ConvertTo-Csv
Note that it's easy to substitute "Import-Csv" and "Export-Csv" to deal with CSV files instead of CSV formatted text.

It's also a trivial extension to convert this snippet into a "ps1" script that can take formal parameters as input from anything that has those two columns, not just CSVs. And those parameters can be invoked manually, validated by the engine (not code!), and so forth.

What I hope this PowerShell version shows off is just how "neat" it is compared to bash. It's just a pipeline of little processing commands: "get,compare,stop,sort,select,convert". As impressive as your bash solution is, it's... messy. You were forced to write manual loops, use JSON for processing CSV, and do all sorts of admittedly pretty fancy trickery. Ask yourself: Could a junior tech make changes to your script and not break it? Could they even read it? I know I certainly couldn't, and I know more than a little bash!

As to how extensible PowerShell is: More than bash!

The "shell API" is just a byte stream in, and a bunch of byte streams out. That's it.

The PowerShell API is very rich, but it's also very easy to get started. You can simply write scripts or write a "binary module" in any .NET language such as C#. Simply derive from the System.Management.Automation.Cmdlet class, override the processing methods as required, and start writing code. The PowerShell engine will take care of things for you like hooking up input parameters, matching wild cards, expanding file paths, validating inputs, generating error messages in the user's OS language, etc...

In my experience, I can write a "proper" command line tool with all the bells & whistles using C/C++ in about a few thousand lines of code. With modern command-line parsing libraries, it's about 500 lines if I'm lucky. Meanwhile, it's possible to build the equivalent using PowerShell and C# in about 50 lines, of which 40 is just generic C# class boilerplate.

Re:Ask HN: Why isn’t visual programming a bigger thing? by JdeBP on Friday June 05, 2020 @03:43AM (#23426025)
It is a stylistic thing, of course, but I tend to put the vertical bars at the ends of the preceding lines, which eliminates the need for escaped newlines.

One of the things not often mentioned about PowerShell is its embeddability. I have explored this since writing https://news.ycombinator.com/item?id=22849569 .

Re:Ask HN: Why isn’t visual programming a bigger thing? by jodrellblank on Friday June 05, 2020 @04:40AM (#23426332)
PowerShell 7 fixes it so you can put pipes on the new line, without needing backtick escape on the previous line. e.g. in Windows Terminal ctrl+enter a newline and write

    gci
    | select name
> One of the things not often mentioned about PowerShell is its embeddability

Not mentioned enough; Exchange 2007 GUI was built with PowerShell integration such that you could click through with a mouse, and it would show you the powershell code for you to copy/paste and reuse it yourself without a GUI. This is a very underused, under-discussed idea, IMHO.

Re:Ask HN: Why isn’t visual programming a bigger thing? by JoshuaDavid on Friday May 22, 2020 @12:38PM (#23273957)
Yeah, the bash ecosystem looks fairly different with regards to dealing with structured data since jq became popular. Basically jq means that as long as you can safely get your input from whatever custom format into json, and from json to whatever output format, you can safely work with structured data.

Incidentally, I think it would have been possible to use only `csvkit` commands and skip the format munging to and from json, but I have an allergy to working with csvs since the format is a bit underspecified and there exist "csv" files that break even the few rules the format is supposed to have. You would be horrified at some of the things I have seen that called themselves "csv".

> In my experience, I can write a "proper" command line tool with all the bells & whistles using C/C++ in about a few thousand lines of code

Yeah, in my experience trying to write command line tools in low level languages is an exercise in pain. I usually write new tools in Python, where similar to your experience writing a simple command is ~20 lines of code, ~10 of which is boilerplate that is the same for every command.

That said, powershell does look pretty cool. And I see that it's possible to install on linux as well[1], and as far as I can tell even to write my commands in python[2], so it almost looks like "a unix-style shell but with types" which would be useful.

[1] https://docs.microsoft.com/en-us/powershell/scripting/instal... [2] http://pythonnet.github.io/

Re:Ask HN: Why isn’t visual programming a bigger thing? by jve on Friday June 05, 2020 @07:14AM (#23427248)
This thread was a joy to read because you guys didn't come into fight :)