News for hackers. Stuff that matters.
Re:Show HN: CLI tool for saving web pages as a single file by gambler on Friday August 23, 2019 @11:18AM (#20778516)
This is a good question. I think you can make it even better by generalizing the problem. How on earth do developers hope to advance general computing forward when simply running programs isn't a solved problem? Most software engineers I know don't run docker on their home PCs. What about people who aren't in IT? Does anyone here even care? The general attitude I see is "plebs don't need to run anything they can't get outside of an app store". It's a horrible attitude.

I very much like this quote from Alan Kay:

"It doesn't matter what the computer can do, if it can't be learned by billions of people."

There is no good technical reason why modern operating systems can't work out some some scheme for sanboxing arbitrary programs by default. It is obviously necessary. I imagine something like "applications" folder where every subfolder automatically becomes an isolated "container". It would have to be designed with security as the primary concern, though; unlike current container solutions.

Re:Show HN: CLI tool for saving web pages as a single file by derefr on Friday August 23, 2019 @12:08PM (#20779009)
But arbitrary programs are... arbitrary. Especially ones run by software engineers, and especially ones run by software engineers as part of a POSIX-alike “utility bag” ecosystem.

Who’s to say that the user’s intent by running the program they just downloaded, isn’t to—say—overwrite a system folder? (Oh, wait, that’s exactly what Homebrew does, with the user’s full intent behind it!)

There are tons of attempts to do what you’re talking about. Canonical’s “snaps” are a good example. As well, every OS sandboxes legacy apps by default (because they’re already virtualizing them, and sandboxing something in a virtualization layer is easy.)

But none of those solutions really work for the “neat FOSS hack script someone wrote” workflow we’re talking about here, where you build programs from source and run them for their intentional side-effects on your system.

You might suggest that there could be a shared sandbox for all the POSIX-like utilities to interoperate in. But what if you’re attempting to use those utilities against your real documents? (For example, a bulk metadata auto-tagging and auto-renaming utility, to get TV episodes from torrents loaded into Plex correctly.) How do you draw the line of what such a program can operate on? AFAICT, you just... can’t. Its whole purpose is to silently automate some task. If it requires constant security prompting, the task isn’t automated.

Re:Show HN: CLI tool for saving web pages as a single file by gambler on Friday August 23, 2019 @01:07PM (#20779781)
>But what if you’re attempting to use those utilities against your real documents?

You copy or move documents inside the specific sandbox.

If you want a pipeline, you establish a chain of inbox/outbox folders.

Obviously, most of this should be done by the OS, not the user.

The workflow:

- You click "download" in your browser.

- When it's done and you click on your download, the OS asks how you want to open it. Instead of "execute" option you get "run in a sandbox" option.

- You type in the name of the sandbox, the app gets copies to /apps/sandboxName or something of that sort.

- The system automatically creates /apps/sandboxName/inbox and /apps/sandboxName/outbox.

- To process a file in some way, you drop it into inbox dir.

For command line, the only change would be switching from "executable pulls arbitrary files" to "I push specific file to the executable".

  zip -r squash.zip dir1
becomes

  | -r squash.zip | zip dir1 |
Start pipeline. Push squash.zip as an argument to zip, get the output. Zip would be the container name.
Re:Show HN: CLI tool for saving web pages as a single file by derefr on Friday August 23, 2019 @02:01PM (#20780495)
Let me put it another way: how would you implement a dotfile management framework (like any of these: https://dotfiles.github.io)? Programmers seem to really like them, judging by how many of them there are. But the whole point of them is to forcefully usurp the assets of literally every other program on the system. They're user-level rootkits, in a sense.

Or, for a simpler, more obvious example: find(1), grep(1), etc. A set of utilities that can all be asked the equivalent of "read literally every file the VFS has access to and tell me whether they match an arbitrary-code-execution predicate." Do you want to literally copy your entire hard disk into the 'inbox' of these utilities, in order to get them to search it for you? (And before you say "well, we can trust the base utilities that ship with the OS to do more than arbitrary third-party utilities"—there's a whole competition of grep(1) replacements, e.g. ag(1), rg(1), etc. Do you want to make it impossible for people to innovate in this space?)

Or how about Nix, or GNU Stow, or, uh, Git? These utilities become useless if they have their own sandbox. Does your git worktree live in Git's sandbox? Vi's sandbox? The inability to make this distinction functional is why mobile OSes only have fullly-integrated IDEs!

Or how about shells themselves! (Or, equivalently, any scripting runtime, e.g. Ruby, Python, etc.) Should people not be allowed to install these from third parties?

Or, the most based example of all: make(1) [and its spiritual descendants], and the GNU autotools built atop it. How does ./configure work if you can't detect true properties of the target system, only of the sandbox you're in?

Re:Show HN: CLI tool for saving web pages as a single file by gambler on Friday August 23, 2019 @03:17PM (#20781478)
>"Do you want to literally copy your entire hard disk into the 'inbox' of these utilities, in order to get them to search it for you?"

Well, let's think about the goal here. grep reads files and outputs lines from those files. It needs full read access to everything you want to search. It does not need write access outside of its sandbox. It does not need direct access to network sockets, audio stuff and so on.

Is it unreasonable to create a readonly "view" of the filesystem inside grep's folder? Is it unreasonable to have "files" representing network access, microphone, audio? It will have visual representation in file manager without the need to create custom UI. It could be manipulated by drag-and-drop OR command line. More importantly: it's easy for users to understand. "This app lives in a box. You can put things in that box for the app to use."

>Does your git worktree live in Git's sandbox?

Yes? I mean, I currently have a folder called projects. All my git stuff is in there anyway.

>Vi's sandbox?

If you want multiple sandboxes to be able to operate on a directory, you create "views" for that directory (readonly or read/write) in multiple sandboxes. This shouldn't be some sort of mind-bending idea, considering Unix has symlinks, hardlinks, and mounted filesystems of all sorts.

>Or how about shells themselves! (Or, equivalently, any scripting runtime, e.g. Ruby, Python, etc.) Should people not be allowed to install these from third parties?

There is no reason why a Ruby executable should have unlimited access to the entire file system. Especially if you're only using it for a specific purpose, like serving a website.

What I'm describing here isn't some novel, mind-blowing idea. It's simply dependency injection. With file-based user interface. Every single part of this had been done in various operating systems or programming environments more than once. It's just a matter of combining it all in a sensible way.

Re:Show HN: CLI tool for saving web pages as a single file by JetSpiegel on Saturday August 24, 2019 @04:22PM (#20789484)
> It does not need write access outside of its sandbox.

You have SElinux for that, if you like bureaucracy and filing triplicate forms to able to run scripts with side effects.

Re:Show HN: CLI tool for saving web pages as a single file by popey on Monday August 26, 2019 @03:31PM (#20802905)
Amusing that you should mention Canonical "snaps". I made a snap of monolith and contributed the yaml upstream. https://snapcraft.io/monolith - it's in the edge channel because upstream haven't done a stable release yet. It's a strictly confined application.
Re:Show HN: CLI tool for saving web pages as a single file by Hackbraten on Friday August 23, 2019 @02:48PM (#20781107)
> Oh, wait, that’s exactly what Homebrew does, with the user’s full intent behind it!

Care to elaborate?

Re:Show HN: CLI tool for saving web pages as a single file by derefr on Friday August 23, 2019 @03:09PM (#20781389)
Homebrew is designed to take over your OS /usr/local directory. Not that there's much in there by default, but Homebrew's presence greatly changes the semantics of /usr/local, given that it's normally meant as a prefix for the local system administrator to install things as root into, whereas Homebrew re-assigns ownership of the whole directory structure to the user installing Homebrew (who, admittedly, is a member of the "staff" group, but still only one member on a potentially multiuser computer.)
Re:Show HN: CLI tool for saving web pages as a single file by skybrian on Friday August 23, 2019 @11:49AM (#20778822)
Even with decent containers, making software that's good enough for everyone to use is considerably more difficult than making it for yourself and a few friends.

It's okay to do things that don't scale at all, and also okay to make a proof of concept and let someone else worry about scaling it up.

That said, you might want to look at Chromebooks and the Windows and Mac app stores to see what's going on with containerization beyond mobile. (Also, web browsers.)

Re:Show HN: CLI tool for saving web pages as a single file by vaer-k on Friday August 23, 2019 @12:47PM (#20779468)
> I imagine something like "applications" folder where every subfolder automatically becomes an isolated "container". It would have to be designed with security as the primary concern, though; unlike current container solutions.

Isn't that basically just Qubes OS? https://www.qubes-os.org/

Re:Show HN: CLI tool for saving web pages as a single file by wongarsu on Friday August 23, 2019 @12:18PM (#20779106)
> I imagine something like "applications" folder where every subfolder automatically becomes an isolated "container"

Snap [1] goes pretty far in this direction. Apps are isolated against each other, and with AppArmor isolated from the system (at least on Ubuntu, your distro might vary). Android does much of the same.

A big problem is that most software exists to manipulate data on the user's machine, so isolating the software from the User folder is impractical. At the same time this data is usually the most valuable thing about the entire computer. That makes it fundamentally very hard to design a system where you can trust arbitrary apps. Android tried to solve this with a "file open" dialog that's controlled by the OS so that there's an easy way to give apps temporary access to single files, but that leads to weird UX.

1: https://snapcraft.io/

Re:Show HN: CLI tool for saving web pages as a single file by Rediscover on Sunday August 25, 2019 @03:03AM (#20791760)
> sanboxing arbitrary programs by default

See OLPC Bitfrost

http://wiki.laptop.org/go/Bitfrost