Yeah, I've been using labview for almost twenty years. I use it because I can hand it to a biologist and have a modest chance they can understand it, plus for me it's easier to make a UI with it since I just never learned that.

But trying to do much in the way of complex logic, unless it's already a built in function, is horrible. Imagine writing a math equation in a visual programming language out of three terminal visual functions for "+". It's hard to be sure you even have the order of operations right! Of course they bandaid this by offering inline equations (which are clunky but I use sometimes) and some support for incorporating textual code for functions, but after twenty years I've never tried to do it because it is more complex than an inline code editor.

It's great for showing people just above the level of excel who need to do something custom and don't know a real language or need a quick GUI. But I guarantee they will screw it up badly since the clean visuals cover a lot of hidden behavior that will screw you up. I usually write them 98% of it since the design is hard to guess a good paradigm for, and let them practice on modifying the remaining 2% as they change their mind about what they want it to do.

I think this hits the nail on the head but I wanted to add a few examples of what I believe to be this effect in practice:

* Roman numerals vs Hindu-Arabic numeral system [1] (aka base 10 number system)

The Roman numeral system is essentially a 'base-1' number system with other symbols attached to represent larger numbers. There are only a finite number of these larger numbers (V, X, C, etc.) so, in effect, the representation of a number grows linearly with the number itself, compared with base-10 which grows logarithmically.

* Logogram [2] vs. Alphabet languages

That is, pictographic languages vs. alphabet-like languages. HJK languages have upwards of 3k [3] pictographs to memorize compared with less than 100 characters for English and other alphabet like languages.

* "Unix vs. Windows"

Maybe a weak comparison, but Unix systems prefer small tools that compose well with each other via pipes using standard text. Unix's focus is on the command line and lends itself to automation. Windows environments focus on single person usability with pictograms, visual aids, etc. Obviously automation can be done in a Windows-like environment (including X) but, as the old adage says, "those who don't understand Unix are condemned to reinvent it, poorly." [4]

[1] https://en.wikipedia.org/wiki/Hindu%E2%80%93Arabic_numeral_s...

[2] https://en.wikipedia.org/wiki/Logogram

[3] https://en.wikipedia.org/wiki/Chinese_characters#Number_of_c...

[4] https://en.wikiquote.org/wiki/Unix

Your overall post is very insightful, but the specific example of Windows is would have been always wrong and especially now is spectacularly outdated.

Windows for decades has had a concept of "object oriented" composability, first through simple message passing, and later through things like the Component Object Model (COM), DCOM, Object Linking and Embedding (OLE). Among other things, this allows quite high level composability, such as Excel graphs in Word documents. This includes system management APIs such as the Windows Management Instrumentation, which is inherently object oriented and has been available for over a decade through the VB or JS shells, and even CMD/Batch shells using "wmic.exe"

Now, UNIX people will argue that this isn't the same as the typical GNU toolset and a shell like bash. They would be right. In Windows, there's a gap between "I can drag and drop an Excel spreadsheet into Word" and "I can use C# to automate COM APIs". It's a big gap. I crossed it, I've seen commercial products that also do various forms of COM automation too, but I can count on one hand people that I have met "in the field" who have ever automated Windows in this manner.

In other words, this gap was wide enough that many people -- especially those most familiar with UNIX -- would often claim that there's nothing on the other side. But this just isn't true.

These days, everyone uses PowerShell, because it bridges that gap. It wraps all of those OO APIs that have existed for a long time in a unified platform. It can natively call WMI APIs, COM, WS-MAN, .NET, and has wrappers for the legacy DCE/RPC management APIs too.

If I can leave you with one takeaway sentence from this is, then it's this one: PowerShell is more UNIX than UNIX.

Seriously. In UNIX, if you want to sort the output of "ps"... sss... that's hard. Sure, it has some built-in sorting capabilities, but they're not a "sort" command, this is a random addon it has accumulated over time. It can order its output by some fields, but not others. It can't do complex sorts, such as "sort by A ascending, then by B descending". To do that, you'd have to resort to parsing its text output and feeding that into an external tool. Ugh.

Heaven help you if you want to sort the output of several different tools by matching parameters. Some may not have built-in sort capability. Some may. They might have different notions of collations or internationalisation.

In PowerShell, no command has built in sort, except for "Sort-Object". There are practically none that do built in grouping, except for "Group-Object". Formatting is external too, with "Format-Table", "Format-List", etc...

So in PowerShell, sorting processes by name is simply:

    ps | sort ProcessName

And never some one-character parameter like it is in UNIX, where every command has different characters for the same concept, depending on who wrote it, when, what order they added features, what conflicting letters they came across, etc...

UNIX commands are more an accident of history than a cohesive, coherent, composable design. PowerShell was designed. It was designed by one person, in one go, and it is beautiful.

The acid test I give UNIX people to see if they really understand how weak the classic bash tools they use is this:

Write me a script that takes a CSV file as an input, finds processes being executed by users given their account names and process names from the input file, and then terminates those processes. Export a report of what processes were terminated, with ISO format dates of when the processes were started and how much memory they used into a CSV sorted by memory usage.

Oh, there's a user called "bash", and some of the CSV input fields may contain multiple lines and the comma character. (correctly stored in a quoted string, of course!)

This kind of thing is trivial in PowerShell. See if you can implement this, correctly in bash, such that you never kill a process that isn't in the input list.

Give it a go.

Thanks for this, a very interesting insight into the Windows ecosystem.

I won't be able to speak with any authority on Windows systems and I certainly don't want to get into a language/OS war but I would like to point out a few issues that raise red flags for me:

* "Powershell is more UNIX than UNIX"

Again, I don't have enough knowledge to really speak on PowerShell specifically but I will point out that the shell is one aspect of a Unix system, even if it's one of the bigger ones. The concept of files, filesystem, memory, in addition to the ecosystem of tools, are part of Unix. Obviously Windows has solved these aspects in some way shape or form but, at least from my perspective of the outside looking in, Microsoft has re-invented a lot of the Unix philosophy in order to make a Windows box behave more like a Unix box.

* The mess of 'simple' tools vs. a cohesive philosophy in Windows

While it's true a lot of the tools in the Unix ecosystem have arbitrary names and non uniform output syntax, the diversity of tools allows for a large range of solutions to a given problem. As you point out, the hodge-podge nature is most likely a historical artifact of the Unix evolution but I would argue this is kind of a natural by-product of systems that are heavily used and improved.

I have no doubt that there are some cleaner interfaces in PowerShell but my concern is the reason for the clean interface is because it's a walled garden with Microsoft as the gate keeper. I think this gets into the old conversation about quality of code for FOSS projects vs. commercial projects.

It's my belief that, after a certain threshold, having a quantity of software to choose from will lead to better quality software overall compared to software that has less quantity but higher proportion of quality. I think this might be a contentious view but my opinion is that it's better to let many "amateurs" experiment, with the best tools from those experiments filtering to the top, than to rely on a few experts to solve the same problems.

Maybe there's a stronger economic argument as it relates to FOSS, I don't know.

I will admit that I have a lot of sunk-cost/first-language-bias that makes me stick with Unix. If Microsoft ever open sources their operating system, I might reconsider that position though.

You can't say that the "hodge-podge nature is most likely a historical artifact of the Unix evolution" and then say that it is "quality". You can't have it both ways. Random, unpredictable, historical quirks do not add up to quality. They add up to a mess.

It's like pointing to a shanty town and saying it is "quality housing" because it has a high population density.

I'm reminded of a recent interview with Jim Keller, who's famous for being one of the architects of the x86 instruction set, Apple's ARM CPUs, AMD's Zen architecture, and Tesla' AI chip: https://www.youtube.com/watch?v=Nb2tebYAaOA

An interesting throwaway quote, but actually a deep insight, is that he thinks a lot of things should be "redesigned every 5 years or so", but unfortunately are redesigned only every decade or longer.

I totally agree. PowerShell was a clean-slate design, and was called "Monad Shell" originally. It's elegant, and is based on a cohesive concept. It's now 10 years old and starting to accumulate... inconsistencies. Warts. It's in need of a clean-slate design again.

Bash and the GNU tools are the most random mess imaginable that can still function. Its components can trace their roots back to the 1960s. Ancient code that should have been rewritten from scratch a dozen times over. Like you said, it's "evolution", but evolution gave us the trigeminal nerve and the inverted retina. I want a designed system that doesn't take the scenic route and have things back to front because it was "always done that way and is too hard to change now".

If you got sat down and told to come up with a set of composable command-line tools for a shell, the end result would have nothing at all in common with the bash/GNU tools. There is nothing there worth repeating.

I've seen that interview before and that quote in particular resonates with me but at the same time there are basic economics to consider.

I want to be clear, what I meant by 'quality' above is functioning systems. If there's a hodge-podge solution that gives you a 10x improvement at the cost of 1/2x the productivity vs. an "elegant" framework that gives you a 2x improvement at 1/10x the productivity by requiring to re-invent all the tools, the 'hodge-podge' solution wins.

To me, this is the same basic argument of Dvorak vs qwerty keyboards. Dvorak might be a 1-2x improvement over qwerty but qwerty wins because of cultural momentum. The improvement isn't worth uprooting the infrastructure already in place.

I think programming languages suffer the same fate, where the size of code base, libraries, etc. secures it as the lingua franca even if there are other languages that are marginally better.

As a rule of thumb, I think a replacement needs to be at 10x the improvement before it has a chance of uprooting entrenched systems.

The Unix system might be chaotic but it's had a lot of time to adapt to different needs. There's also a culture of freedom, sharing and experimentation that isn't present in the Windows world. These all have real world implications on what tools are available for experimentation, use and improvement.

> If you got sat down and told to come up with a set of composable command-line tools for a shell, the end result would have nothing at all in common with the bash/GNU tools. There is nothing there worth repeating.

Sorry, no. This sounds like an emotional plea rather than anything rooted in critical thinking. Sorting text files isn't worth repeating? Searching text files isn't worth repeating? Piping text from one process to another isn't worth repeating? I've tried to be even keeled in my responses but this is ridiculous.

> I have no doubt that there are some cleaner interfaces in PowerShell but my concern is the reason for the clean interface is because it's a walled garden with Microsoft as the gate keeper

This is simply not true. Many third-parties have powershell modules to interact with their product. Anyone can write them.

> UNIX commands are more an accident of history than a cohesive, coherent, composable design. PowerShell was designed. It was designed by one person, in one go, and it is beautiful.

I'm curious about that design and would love to learn about it.

Note, that I'm not interested in learning how to actually use powershell as a developer, but understand the abstraction design choices, that went into it.

Every time I tried to learn about that, I'd find recommendations about Books with 1000+ pages and similarly time consuming resources.

Do you have an recommended read for understanding the design choices, that went into powershell, while not learning the details, that a everyday windows would need?

Things I care about: -How do Lamdas work? -Arrays are Objects. How exactly? -Typesystem -Are there native wrappers for the kernel APIs how do they fit into the objectmodel? -Since to my understanding no shortflags exist, how is verbosity avoided? -How thight are interfaces defined. If a tool provides a new interface in a new version, can old tools still work with it?

There are no good references other than a handful of blogs from over a decade ago. Google "Monad Shell", which was the original name during which the formative ideas were still congealing.

Generally, PowerShell simply embraces the UNIX concept, but with a clean slate. Being based on the .NET runtime, it inherits much of its low-level functionality, such as being able to call into WMI, COM+, and Win32 APIs.

But what it's really about is keeping things in strongly typed objects until the last second when they are formatted for display. The original UNIX pipes formatted output immediately, because C is not object oriented. So everything ends up as a byte/char stream, and then to glue commands together you often have to parse & split the output. There's ways around this, but it's inconsistent and not at all general.

So whereas in bash you would see only 2 or 3 commands in a row with a pipe symbol in between them, in PowerShell you would often be able to chain 10 or even 20 commands in a row in a single pipe. You build up pipelines almost like SQL queries, which is easy because the data passing between each step is structured.

The other big win that I see over legacy shells that got built up over decades is the discoverability and consistency. The naming is consistent, and all the commands publish detailed metadata that integrates with both the help system and tab complete.

Issues like verbosity are simply sidestepped. PowerShell is verbose to read, which makes it readable. Typing it is fast, because you can tab complete not just the command names, but the parameter names, and even the parameter values in most cases. No other shell can do this. It has aliases, so if you prefer 'ps', you can use that instead of the canonical 'Get-Process' command name.

Generally all commands simply extend a .NET class called "Cmdlet", which defines functions such as Begin(), Process(), and End() that handle pipeline input. Declarative parameters can be defined, which the shell engine hooks up for you automatically. For calling external APIs, you can do whatever .NET can do, which is pretty much everything in Windows and Linux with only a handful of exceptions (no inline assembly language).

In terms of interfaces, it's a little bit too weakly typed for my taste. When binding parameters or pipelines, the engine goes through a complex process of figuring out what you meant. It can go wrong. The classic example is that some search or query commands can either return a single object or an array, and making this consistent is a bit of a pain.

It supports lambdas, they're simply a block of code with squiggly brackets around them, e.g.: { echo "foo" }. You can do all sorts of fun things with them, like capture a closure with simply: { ... }.GetNewClosure()

Install PowerShell 7 on Linux, type in {}. and press tab to cycle through the list of functions you can call on an expression block. I like toying around with things like .Ast. The engine exposes some crazy powerful things directly, such as being able to generate code for remote RPC wrappers automatically, etc...

Yes, it's not particularly difficult either, as long as you have the appropriate tools installed.

    csvjson path/to/infile.csv \
        | jq -c ".[]" \
        | while read LINE; do
            UNAME="$(echo "$LINE" | jq -r ".uname")";
            CNAME="$(echo "$LINE" | jq -r ".cname")";
            if id -u "$UNAME" &> /dev/null; then
                # Find processes with this command name belonging to this user
                comm -12 <(ps -C "$CNAME" -o "pid=" | sort) <(ps -U "$UNAME" -o "pid=" | sort) | \
                    while read PID; do
                        # rss=Resident Set Size, i.e. the amount of memory this process is currently
                        # using, not including swap. You might want vsize instead, depending on your needs.
                        RSS="$(ps -o "rss=" "$PID")";
                        STARTISODATE="$(date --date="$(ps -o "lstart=" "$PID")" --iso-8601="seconds")";
                        # If you actually plan to use this you need to make sure pid isn't anything important
                        kill "$PID";
                        jq -n -c \
                            --argjson "pid" "$PID" \
                            --argjson "rss" "$RSS" \
                            --arg "start" "$STARTISODATE" \
                            --arg "command" "$CNAME" \
                            --arg "username" "$UNAME" \
                            '{"pid":$pid,"memory_kb":$rss,"start":$start,"command":$command,"username":$username}';
                    done
            else
                # Print an error to stdout if the user does not exist
                echo "warning: user $UNAME does not exist" >&2
            fi
        done \
        | jq -s 'sort_by(.memory_kb)' \ # sort_by(-.memory_kb) if you want it in descending order
        | in2csv --format=json \
        > path/to/outfile.csv

I'm guessing you meant "parse the CSV correctly using only bash", and I agree that's difficult, which is why you should not do that. Instead, you should use a tool that has been specifically designed to do that one task, and do it well (in this case, that task is "convert a csv into a format that's easier to work with, namely json"). This is the unix way.

How extensible is PowerShell? With bash, you can write a new command that takes text input in a certain format and produces text output in a certain format, in whatever language you want, and plug it in in the appropriate place in the pipeline. Is the same true of PowerShell?

I am properly impressed. Wow!

Most people I give this challenge to fail to account for the complexities of even a simple format like CSV, and typically simply use "grep" to find processes, which would cause havoc with simple user names like "Bob Ash = bash".

For comparison, the equivalent in PowerShell is this:

    #require -RunAsAdministrator
    $i = ConvertFrom-Csv @'
    UserName,ProcessName
    DOMAIN\BobJones,explorer
    DOMAIN\Bash,notepad
    '@
    
    $ps = Get-Process -IncludeUserName
    Compare-Object $ps $i -PassThru -Property UserName,ProcessName -ExcludeDifferent -IncludeEqual  `
        | Stop-Process -PassThru `
        | Sort-Object PeakWorkingSet64 -Descending `
        | Select-Object `
            ProcessName, `
            PeakWorkingSet64,`
            @{ Name='StartDate'; Expression={ '{0:o}' -f $_.StartTime } } `
        | ConvertTo-Csv

Note that it's easy to substitute "Import-Csv" and "Export-Csv" to deal with CSV files instead of CSV formatted text.

It's also a trivial extension to convert this snippet into a "ps1" script that can take formal parameters as input from anything that has those two columns, not just CSVs. And those parameters can be invoked manually, validated by the engine (not code!), and so forth.

What I hope this PowerShell version shows off is just how "neat" it is compared to bash. It's just a pipeline of little processing commands: "get,compare,stop,sort,select,convert". As impressive as your bash solution is, it's... messy. You were forced to write manual loops, use JSON for processing CSV, and do all sorts of admittedly pretty fancy trickery. Ask yourself: Could a junior tech make changes to your script and not break it? Could they even read it? I know I certainly couldn't, and I know more than a little bash!

As to how extensible PowerShell is: More than bash!

The "shell API" is just a byte stream in, and a bunch of byte streams out. That's it.

The PowerShell API is very rich, but it's also very easy to get started. You can simply write scripts or write a "binary module" in any .NET language such as C#. Simply derive from the System.Management.Automation.Cmdlet class, override the processing methods as required, and start writing code. The PowerShell engine will take care of things for you like hooking up input parameters, matching wild cards, expanding file paths, validating inputs, generating error messages in the user's OS language, etc...

In my experience, I can write a "proper" command line tool with all the bells & whistles using C/C++ in about a few thousand lines of code. With modern command-line parsing libraries, it's about 500 lines if I'm lucky. Meanwhile, it's possible to build the equivalent using PowerShell and C# in about 50 lines, of which 40 is just generic C# class boilerplate.

It is a stylistic thing, of course, but I tend to put the vertical bars at the ends of the preceding lines, which eliminates the need for escaped newlines.

One of the things not often mentioned about PowerShell is its embeddability. I have explored this since writing https://news.ycombinator.com/item?id=22849569 .

PowerShell 7 fixes it so you can put pipes on the new line, without needing backtick escape on the previous line. e.g. in Windows Terminal ctrl+enter a newline and write

    gci
    | select name

> One of the things not often mentioned about PowerShell is its embeddability

Not mentioned enough; Exchange 2007 GUI was built with PowerShell integration such that you could click through with a mouse, and it would show you the powershell code for you to copy/paste and reuse it yourself without a GUI. This is a very underused, under-discussed idea, IMHO.

Yeah, the bash ecosystem looks fairly different with regards to dealing with structured data since jq became popular. Basically jq means that as long as you can safely get your input from whatever custom format into json, and from json to whatever output format, you can safely work with structured data.

Incidentally, I think it would have been possible to use only `csvkit` commands and skip the format munging to and from json, but I have an allergy to working with csvs since the format is a bit underspecified and there exist "csv" files that break even the few rules the format is supposed to have. You would be horrified at some of the things I have seen that called themselves "csv".

> In my experience, I can write a "proper" command line tool with all the bells & whistles using C/C++ in about a few thousand lines of code

Yeah, in my experience trying to write command line tools in low level languages is an exercise in pain. I usually write new tools in Python, where similar to your experience writing a simple command is ~20 lines of code, ~10 of which is boilerplate that is the same for every command.

That said, powershell does look pretty cool. And I see that it's possible to install on linux as well[1], and as far as I can tell even to write my commands in python[2], so it almost looks like "a unix-style shell but with types" which would be useful.

[1] https://docs.microsoft.com/en-us/powershell/scripting/instal... [2] http://pythonnet.github.io/

This thread was a joy to read because you guys didn't come into fight :)

Most visual programming is two-dimensional. What if it were three-dimensional instead, arranged in some sort of https://en.wikipedia.org/wiki/Axonometric_projection with https://en.wikipedia.org/wiki/Platonic_solid 's representing your modules/nodes, chosen according to the amount of connections they have with other modules/nodes, the connections styled and color coded, maybe even unobtrusive/discreetly animated to show the direction/speed/volume of signals/dataflow, width of datatypes/buses, all in an abstract fashion similar to the way large metro/rapid-transit systems are mapped?

Able to encapsulate other modules/nodes, folding large graphs into one, popping into an https://en.wikipedia.org/wiki/Exploded-view_drawing only on demand?

Turnable/viewable from any direction you like, but usually snapping to some virtual 3D-grid, in say steps of 45° ?

Zoomable?

This is even usable the other way around for reasoning about stuff you have NO source code for. Like in https://www.youtube.com/watch?v=4bM3Gut1hIk (skip to about 14m30sec, the show starts there) and https://sites.google.com/site/xxcantorxdustxx (same guy) and others, but similar https://codisec.com/binary-data-visualization and even more so https://arcan-fe.com

That looks messy, but that is because it is in reverse, with no known sources.

Now imagine how tidy it could look when done originally, from first principles.

Your own cybernetic Zettelkasten/Memex/Dynabook/whatever in a world of common and open (architecture neutral) data & document formats, automagically tagging, indexing and sorting all your stuff.

OFFLINE if you like to.

Usable on conventional desktops, tablets, large surfaces, overlaid onto some augmented reality, VR, holographics, i don't care.

With the end goal of partially dynamic reconfiguration/JIT of some fpga-like logic fabric, according to what you click, tap, swipe in that environment, while hosting itself.

Without crashing. Formally verified at the same time.

I was also thinking about this the other day. Essentially we should be able to take advantage of the brain's superb ability to comprehend/remember locations in our day to day programming experience. This would lower the cognitive load of translating the codebase into a mental model, since locations are easier to remember (memory palace) and reason about.

On a related note, the Unison language [1] and its data structure of the codebase lends itself perfectly for this kind of visualization (since you get the dependency graph between functions for free). Imagine being able to see the whole codebase as some kind of large, coarsely detailed factory/assembly line, linking all your higher-level functions together. Only when you "get" sufficiently close, you can make out the inner connections between the functions.

[1] https://www.unisonweb.org/ - a programming language where the definitions themselves are content-addressed

Hm. I remember reading about it, but it isn't bookmarked. So i probably missed something. What i didn't write in my post, was different symbology at different levels of zoom. But you got the general idea, and I what you probably mean.

edit: Would maybe look nice, but not very useful, to only have platonic solids, however they are connected. I'm thinking of icons/thumbnails also, like in airports, stations, symbolising data types, (line)encoding schemes, combined with something like https://en.wikipedia.org/wiki/Shunting-yard_algorithm / https://en.wikipedia.org/wiki/DRAKON depending on context/zoom level and simply NOT fitting to each other, if you try to couple them in incompatible/unsafe ways.

2nd edit: the same for common algorithms, have some box/body, define the range/width of possible inputs via some sliders/knobs, which change the possible plugs, ready.

I had dreams about this ~2006, FWIW.

I think there's something to be done there, even if it's only for visualizing existing textual code, via some kind of VR, using human spatial memory and intuitions to better effect than tabs and treeviews and back buttons for navigating files and code structure.