News for hackers. Stuff that matters.
Re:Shfmt – format shell programs (like gofmt, rustfmt) by VWWHFSfQ on Saturday February 11, 2023 @01:34PM (#34754895)
The caveats are somewhat of a deal breaker unfortunately. It fails on perfectly valid syntax because they don't want to complicate the parser. But that makes the tool not very useful.
Re:Shfmt – format shell programs (like gofmt, rustfmt) by Spivak on Saturday February 11, 2023 @01:38PM (#34754946)
I can’t say I’ve ever used the forms they don’t support and I am the person who uses all sorts of esoteric bash features at work.

Also for static typing an analysis I would absolutely give up even more syntax that is ambiguously parsed.

Re:Shfmt – format shell programs (like gofmt, rustfmt) by jamal-kumar on Saturday February 11, 2023 @01:52PM (#34755084)
I throw all my shell scripts through this beast of a haskell application to see if they're clean:

https://github.com/koalaman/shellcheck

Crucially it shows where on the line the error is in case I've got some large piped one-liner which might have a problem.

Re:Shfmt – format shell programs (like gofmt, rustfmt) by 2h on Saturday February 11, 2023 @01:49PM (#34755052)
Have to agree with the other response. Demanding perfection is not realistic in many situations. If you can get 95% of the way there with 50% of the code/effort, you should do it. Sometimes being productive is knowing when that last percentage just isn't worth it.
Re:Shfmt – format shell programs (like gofmt, rustfmt) by mvdan on Saturday February 11, 2023 @01:49PM (#34755059)
For the first two caveats, I actually agree that we could and should handle ambiguous input. It just hasn't been a priority because doing that properly would be quite a bit of work, and such ambiguous syntax isn't particularly common. See https://github.com/mvdan/sh/issues/686 for my current thoughts on how to tackle it.

The third caveat concerns parsing `export` and `let` as keywords rather than as builtins. Like the README says, this is to properly build the syntax tree without leaving opaque strings as expressions, but also to support `declare foo=(bar)` which wouldn't work if `declare` was treated like any other builtin simple command.

How else would you have a static parser handle these two builtins? They are in a bit of an awkward middle ground between builtin and keyword. My instinct is that giving them special treatment in the parser to allow tokens like `(`, while at the same time representing them in the syntax tree with opaque strings as expressions, would be pretty underwhelming to any users of the parser.

That said, we already have that problem with `let "foo=123"` for example, where our parser currently represents the expression as the quoted string without going any deeper. https://github.com/mvdan/sh/issues/754#issuecomment-96329574... considers doing a second parse stage in the shell interpreter to fix cases like these, though always doing a second parse could get expensive.

We _could_ leave all arithmetic expressions as input strings in the parser, and do all the actual parsing when they are evaluated. That would be more compatible with Bash and more consistent. But it would also be less useful to any parser users who don't run into any of these weird edge cases, which aren't common at all, I think.

In short, I have some ideas, but I'm not sure at all what's best :) Doing a good job for 99% of users feels better than aiming for 100% compatibility with bash syntax, particularly where bash syntax is a bit weird.

Re:Shfmt – format shell programs (like gofmt, rustfmt) by auveair on Saturday February 11, 2023 @05:43PM (#34757179)
Thank you for taking the time to answer, as a random user it was illuminating.
Re:Shfmt – format shell programs (like gofmt, rustfmt) by usr1106 on Saturday February 11, 2023 @03:55PM (#34756237)
We have used it for more than a year in CI at work. We have many dozens of scripts from various coders. Nobody has ever complained that those caveats would have affected them. I was not aware of them, needed to search now where they even are (need to follow the more info link).
Re:Shfmt – format shell programs (like gofmt, rustfmt) by throwawaaarrgh on Sunday February 12, 2023 @01:31AM (#34760447)
you could always do

  shfmt foo.sh || true
worst case it just doesn't format it?