News for hackers. Stuff that matters.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @12:01PM (#18293590)
Right, that's one problem. The author says:

> But you'd be careful writing "rm" anywhere in Linux anyway, no? Also, why would you want to pipe something into "rm"?

But the thing is that you don't need to intend to pipe to "rm". Maybe you were typing something else, like having a command "rmore" or something. This danger is also not strictly limited to `rm` and `dd`, and you have to be careful that no substrings from the start of the command you intend to write cannot be interpreted as something dangerous.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by wpietri on Wednesday October 24, 2018 @01:09PM (#18294318)
The author's theory there about user behavior seems dangerously wrong.

I'm quite careful writing command lines because the whole experience of the shell is that of working with sharp knives: you get a lot of power but if you screw up, you'll feel the pain. The point of this tool is to take away a lot of the pain that teaches people caution. Its whole theory is "just try a lot stuff as you explore what you want".

In their shoes I'd look at using some of the container/security magic as a way of nerfing commands. If the on-a-keypress runs work in a way where they can't make changes to a filesystem, that seems way better to me. Even better if the tool then reports, "Would have deleted 532 files in a warning color at the top of the output."

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by groestl on Wednesday October 24, 2018 @03:02PM (#18295351)
> I'm quite careful writing command lines

A safety measure I picked up from a sysadmin while watching over their shoulder: start writing nifty command lines by prefixing them with # first, to prevent havoc when fat fingering the [enter].

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by mixmastamyk on Wednesday October 24, 2018 @03:33PM (#18295613)
Yes, though I prefer: echo ...

The reason is that you'll see expanded variables and wildcards before commiting to them.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by rcthompson on Wednesday October 24, 2018 @03:46PM (#18295719)
But prefixing with echo only disables the first command in a pipeline, while commenting it disables the whole thing.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by groestl on Wednesday October 24, 2018 @03:56PM (#18295806)
It also executes command and process substitutions, opens files etc. But granted, it's a useful trick in some cases, after passing the fat-fingering stage.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by mixmastamyk on Wednesday October 24, 2018 @06:36PM (#18297061)
You can move the echo to the part you are currently working on.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by yread on Wednesday October 24, 2018 @04:09PM (#18295892)
It's a pity you can't do something similar to SQL: when writing UPDATE/DELETE always write the WHERE clause first
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by rmetzler on Wednesday October 24, 2018 @04:16PM (#18295946)
Something I learned early on when writing possible dangerous SQL selects:

  CREATE TABLE mytable_backup AS SELECT * FROM mytable;
  SELECT * FROM mytable WHERE condition;
  DELETE FROM mytable WHERE condition;
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by traviscj on Thursday October 25, 2018 @03:41AM (#18299188)
Or just let the DB work for you with BEGIN; + (select/update/delete) + COMMIT; or ROLLBACK;
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @04:18PM (#18295960)
What I like about SQL is that it's got the double safety of ";". To accidentally run a SQL statement before it's ready in a command line, you'd have to both add the ";" AND hit Enter, or have the bad practice of adding ";" before the statement is ready, or use a bad SQL command line that sees the ";" as optional.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by groestl on Wednesday October 24, 2018 @05:00PM (#18296339)
I always start a session with BEGIN, write UPDATEs/DELETEs as SELECTs first and (if possible) use a staging database until I'm fine with the result.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by amalcon on Wednesday October 24, 2018 @10:49PM (#18298263)
In SQL, you can at least wrap the whole thing in a transaction. Then, just roll the whole thing back if anything came out wrong.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by inopinatus on Thursday October 25, 2018 @02:59AM (#18299024)
I usually SELECT COUNT(*) WHERE ..., and only subsequently overwrite with DELETE.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by androidgirl on Wednesday October 24, 2018 @03:52PM (#18295778)
Thanks for the tip, I'll definitely be using this. Fear of accidental rm -rf * keeps me up sometimes.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by samstave on Wednesday October 24, 2018 @05:18PM (#18296495)
Looks like you need to know about the 'sleep' command...
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by yesbabyyes on Wednesday October 24, 2018 @05:36PM (#18296644)
unzip; strip; touch; finger; grep; mount; fsck; more; yes; fsck; fsck; umount; sleep
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jxy on Wednesday October 24, 2018 @03:12PM (#18295463)
`fc` helps. Setting `FCEDIT=ed` helps a lot.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by no_news_is on Thursday October 25, 2018 @10:53PM (#18306389)
This command can be dangerous itself.

For those who don't know... "The fc utility lists or edits and reexecutes, commands previously entered to an interactive sh."

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by clhodapp on Wednesday October 24, 2018 @12:17PM (#18293775)
It seems like most of the scariness could go away if only you had to press a key combo each time you wanted to run the command / render the preview
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @01:02PM (#18294239)
Yes. Like Enter? People might still like the fact that the cursor doesn't need to move from wherever it's editing the command to run it, and can still continue to edit there.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @01:07PM (#18294294)
That's the main direction I currently seem to be coalescing on based on feedback I'm getting. That said, one person on lobste.rs suggested trying to play with Linux capabilities, to make the filesystem immutable... now that would be a seriously fancy trick if it works!
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by curun1r on Wednesday October 24, 2018 @02:47PM (#18295218)
Can I say that I hope you don't make this tool too Linux-specific. It seems to run pretty reasonably on Mac and would probably also work on BSD variants. Since I use all three (Mac, BSD, Linux) on a semi-regular basis, I'm pretty unlikely to integrate a tool into my Unix workflows if it doesn't work reliably on most unixes, so if you can look into cross-platform ways to accomplish things, that would widen the audience of people who could get value from the tool.

For example, one way to address the rm problem would be to give it a "safe mode" that drops user permissions and runs as nobody. You might also create a blacklist of common, potentially-dangerous unix commands that you don't execute. Both of those are examples of how to address the need without resorting to Linux-only tricks.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @05:08PM (#18296410)
How can I do the "drop user permissions" thing? I haven't seen such a keyword mentioned by anyone yet, sounds interesting. Does this not require some kind of syscall, however, thus not really being cross-platform anyway?
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by laumars on Wednesday October 24, 2018 @06:22PM (#18296990)
‘nobody’ is an only UNIX thing. It’s just a user and group (like root, $USER, etc) except ‘nobody’ is meant to have no permissions to any folders nor files.

In practice it’s only as good as your users ability to avoid the temptation of ‘chmod 777’ (for example) but it seems a good place to start.

As an aside, I had the same idea to do a tool like this as well. Except mine would have been built into the $SHELL I’m writing (as that has a heavy focus on being more IDE-like than traditional shells). I scraped the plan for pipe previews precisely because of the dangers we’re discussing. However your tool makes a lot more sense because at least that is manually invoked whereas my original plan was to have that feature automatic and baked into the shell - which is an order of magnitude more dangerous. So I’m glad someone else has ran with this idea

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @07:01PM (#18297214)
:)

I've seen somebody mention the idea of applying this to a shell, but only after pressing some special key (e.g. Ctrl-Enter). Would probably make it un-dangerous enough? This seems to be what people want of up too, anyway.

As to the "nobody", from what I'm reading, it seems you'd first have to be root, to be able to switch to "nobody"... so this doesn't really seem to be useful to me in this case... :/

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by curun1r on Wednesday October 24, 2018 @10:36PM (#18298198)
The other way to do it is to change ownership of the executable to nobody:nogroup and set the setuid/setgid bits.

Perhaps you could simply put those chown/chmod commands in the docs:

    sudo chown nobody:nogroup path_to_up
    sudo chmod ug+s path_to_up
I've tested it, and it seems to prevent deleting files with rm. What doesn't work, however, is that it also prevents writing the results to up1.sh. Perhaps if writing to the file fails (or you detect the process is running as nobody), you could send the finished pipe sequence to stdout instead of a shell script. Then, people could run it like:

    cmd | up > up1.sh
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by laumars on Thursday October 25, 2018 @02:18AM (#18298869)
The solution there is not to set any writable bits for the up executable. Then only root will be able to write to it (which is ideally what you want for any tools within /usr/bin (whatever) anyway
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by laumars on Thursday October 25, 2018 @02:28AM (#18298914)
That’s an interesting argument having it hot-key triggered. The shell in its current form already supports hit-keys so I could plug right into your tool verbatim that way. The only issue is you call fork shell so any $SHELL specific behaviours of my shell would be lost.

I appreciate this is a personal project and sometimes there is nothing more annoying than having feature requests; but if you did ever decide to add a flag for choosing alternative shells then drop me a message ( raise it as an issue on github.com/lmorg/murex ) and I’ll add ‘up’ as an optional 3rd party plug in.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by em-bee on Wednesday October 24, 2018 @07:27PM (#18297359)
i fear a blacklist won't be enough. better a user configurable whitelist. every time a command is not on the whitelist, don't run it, but give the user the option to add it to the list instead (with a multi key action, so that it can't be added to easily and by accident)

greetings, eMBee

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by rcthompson on Wednesday October 24, 2018 @01:13PM (#18294357)
Consider that the filesystem isn't necessarily the only thing you can mess up by accident. It's much more unlikely, but you could accidentally trigger a bunch of unwanted network requests (e.g. piping into `xargs curl` or something), among other possible things with external effects.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @02:01PM (#18294820)
Maybe cutting off network access could be done too? That would reduce some potentially useful features, but it could be a parameter to enable it (hey, we're talking Linux, never enough parameters! ;). That is, assuming that a syscalls/capabilities trick like that is possible at all. (Anyone can say more here? Some kernel hackers? This is HN, ain't it?)
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by rcthompson on Wednesday October 24, 2018 @02:15PM (#18294934)
Yeah, I think almost any ability to produce an effect other than printing to stdout and stderr should probably be restricted by default, with a flag to enable it. If people adopt this and frequently use specific capabilities, they can add a shell alias to add enable their preferred set of capabilities.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by rcthompson on Wednesday October 24, 2018 @03:38PM (#18295649)
Or, one possible alternative would be to always prevent any side-effects other than printing output, and then display a warning saying something like "This command tried to access the network/modify the filesystem/be naughty in some other way. To allow this and re-run the command, press Control+Shift+Enter"
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by em-bee on Wednesday October 24, 2018 @07:32PM (#18297392)
capabilities should certainly work. for portability, setting a non-existant proxy should also help. most commandline tools should honor shell proxy variables and fail the request.

greetings, eMBee.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by deelowe on Wednesday October 24, 2018 @01:50PM (#18294709)
I think from a design perspective, it's better to build your sanity checks and safeguards into the UI instead of some fancy abstraction that's running in the background. Throw in some options to make it user configurable and you have a pretty slick solution that behaves in a way that'll feel familiar to gurus.

I think requiring a keypress by default to run any commands and maybe even throwing up a warning for a list of known dangerous ones (perhaps user configurable) would work well. A really fancy solution would be to do this stuff via some sort of linting.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @02:30PM (#18295088)
I don't see how linting could work, I'm afraid. There are obscure options in zip, git, grep, and what else, not to mention full-blown interpreters like awk and perl, that can edit files via a myriad of syntaxes, and then there's the halting problem. Similarly with whitelist/blacklist, I'm not currently really convinced it could help much; take `xargs rm`, vs. some hyphotetical `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?) Keypress, on the other hand, sounds to me like it could be a reasonable default compromise... still with an option to switch to "fully accelerated, fast mowing" mode at any point, if one so desires...
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Thursday October 25, 2018 @08:35PM (#18305857)
> `foobar -rm` command + params (where maybe -r = recursive, -m = monitor?)

An example of such a foobar with those exact options would be `inotifywait`. :)

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by deelowe on Wednesday October 24, 2018 @05:05PM (#18296386)
Agreed. As another commenter said KISS.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @01:17PM (#18294386)
Too fancy in my opinion. Personally, I prefer KISS solutions. Having an immutable filesystem in up would be unexpected, and I'm sure there's other destructive things that can be done that don't involve the filesystem. As odd as it may seem, I may also want to be able to do filesystem destructive things in up, why shouldn't I be able too?

EDIT: As an example, I may exploratively be playing with find options to match different sets of files in a directory hierarchy. Once I see I've matched the files I want, I may want to add a `-delete` to the end of the find conditions I specified to delete them all. That seems like a useful use of up, but making the filesystem immutable would disallow it.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @01:51PM (#18294712)
Hm. I think in my ideal vision, you'd then quit up, and somehow magically have the pipeline ready at your fingertips on the shell prompt, so that you can just add the `-delete` there in a "regular" fashion, and press Enter. Not quite sure how this could be done with bash easily. I think the upN.sh file is not a bad approximation however. (Someone on reddit (I think) seemed to mention that zsh seems to have some functionality which could allow building something like that over up.) Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access? But even if I add some safety options, even if I make them the defaults, I'll try really hard to keep an option available for people who want to play with it raw, with no safety belts.

edit: I totally get you as far as KISS goes; that's why I built and shared the tool as is in the first place ;)

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @02:17PM (#18294944)
> Interestingly, this seems to me to support the idea that limiting syscalls/capabilities could be a good approach (if at all possible, that is). Potentially maybe also cutting off network access?

Yeah, I still think it's not a good approach. rcthompson gave an example with network access, but that doesn't mean that bad things can only happen with filesystems or network access. You can also kill processes, shutdown the computer, and many more things. I don't think you can guess what mistakes the user might do, or which were really mistakes or were intentional.

Also, if you can avoid special permissions or capabilities for your core functionality, then that's better. People should be conservative in giving special permissions to programs, and I can't think it makes sense to give "up" the ability to make a filesystem immutable even if it's in an isolated namespace.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @02:23PM (#18295008)
Hmmm; so what would you think of the other popular suggestion, to add a (probably default) mode, where only after pressing, say, Ctrl-Enter, the command would be executed? With another shortcut allowing to switch back and forth to the "no seatbelts, confident, fast moving, fast mowing hacker" mode?
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by jolmg on Wednesday October 24, 2018 @02:59PM (#18295335)
I would just do the Enter thing (I can't think of a reason to prefer Ctrl-Enter), and forget about the other mode. If you really want to do that other mode with a super-restricted environment, just have the code prepared to be denied permission (to setup that environment) and not assume that it has been granted. There will be users that won't feel comfortable adding those permissions/capabilities to "up".
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by em-bee on Wednesday October 24, 2018 @07:36PM (#18297408)
you are building up a pipeline, you don't want it to be destructive until you are done.

so at the very end you could apply a special action that now runs the resulting command with safety disabled.

or toggle between safe and unsafe mode at will, with safe mode being default.

it's not necessary to have the whole up utility run with filesystem disabled. but it could apply the restriction only to the pipeline its executing. that would allow it to selectively apply the restriction as requested by the user.

greetings, eMBee.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by twanvl on Wednesday October 24, 2018 @01:45PM (#18294678)
Another solution is to only allow commands from a whitelist to be run. If the user types something not on the list you say "rm is not on the list of allowed commands, press enter to run and add it".
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by ElijahLynn on Wednesday October 24, 2018 @03:06PM (#18295398)
++

And rather a blacklist, since a blacklist would be much smaller than a potentially an infinite amount of programs to manage for a whitelist.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by chrisshroba on Wednesday October 24, 2018 @03:32PM (#18295607)
I think the whitelist makes more sense, since you can never say for sure that you've added every potentially harmful program to the blacklist. If it's easy to add things to the whitelist, as the parent comment proposed, I think the idea would work very well, especially considering most people have a relatively small number of commands they pipe other commands into.

Btw, if you want to see all the commands you've ever piped data into, sorted by frequency, you can use this command (which I put together using `up`!)

    history | grep -o '| \w\w*' | sort | uniq -c | sort -n | nl
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by Spakman on Wednesday October 24, 2018 @02:06PM (#18294865)
Another thing you might want to investigate (for Linux, not sure about cross-platform alternatives) is OverlayFS[1], which allows you to create union filesystems where some layers are read-only and the top layer is read-write. This can allow destructive operations like rm without actually removing anything from the lower (read-only) layers, using whiteouts.

I'd have replied to this on lobste.rs but don't have an account. I'm not a heavy poster but I'd love an invite if anyone has one going :)

1 - https://www.kernel.org/doc/Documentation/filesystems/overlay...

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @02:14PM (#18294921)
I need your email for an invitation; if you can send me your address via my own email (see my profile), or via keybase, I'll send you an invite. I don't see your email advertised in your profile here on HN, so my hands are tied for now :)
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by Spakman on Wednesday October 24, 2018 @02:26PM (#18295037)
Thank you very much! I hadn't realised my email address isn't visible here. Maybe I need to move it into the "about" section.

In case you aren't aware, your email address isn't visible in your profile either. I've emailed the GMail address that I found on your website :)

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by taeric on Wednesday October 24, 2018 @01:44PM (#18294666)
Eshell can be setup to do this. Is neat for small commands. I turned it off, though.
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by akavel on Wednesday October 24, 2018 @01:58PM (#18294786)
Can you elaborate a bit more? Which functionality are you referring to? And why did you disable it? Really curious! (Author of up here) I get it that by eshell you mean a shell in Emacs? (That's at least what I'm getting as a first result from google)
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by taeric on Wednesday October 24, 2018 @02:48PM (#18295235)
Indeed, I meant the emacs shell. https://www.masteringemacs.org/article/complete-guide-master... has a good overview in the section regarding the Plan 9 Smart Shell.

I ultimately just didn't find I used it much. If i am iterating, org-mode with shell sources fits a bit more natural for me.

Edit: and to be clear, I meant specifically to have the output below the command with the command still in edit mode. Was not trying to say it is the same as up.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by shittyadmin on Wednesday October 24, 2018 @12:25PM (#18293848)
Well, that's only completely horrific. Cool idea, but should definitely be limited to a fixed list of commands or something. Even then, some stuff that might be handy like xargs could be quite dangerous...
Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by woodruffw on Wednesday October 24, 2018 @06:20PM (#18296980)
> But you'd be careful writing "rm" anywhere in Linux anyway, no? Also, why would you want to pipe something into "rm"?

Not exactly a pipe into `rm`, but `foo | xargs rm` is a common enough pattern.

Re:Show HN: Ultimate Plumber – a tool for writing Linux pipe... by saagarjha on Wednesday October 24, 2018 @07:27PM (#18297362)
It’s also dangerous if your filenames have spaces in them.