News for hackers. Stuff that matters.
Re:Subuser turns Docker containers into normal Linux program... by timthelion on Friday February 12, 2016 @10:09AM (#11087658)
Subuser is not a revolutionary peice of software. You are right that it is a LOT like a zip file or tar.gz file full of bundled dependencies. It is, however, a big improvement on just that. Subuser contains the subusers, so they cannot mess with the rest of your system, making a lot eisier to trust those bundled dependencies. It also provides some rather primitive update mechanisms for those bundled dependencies. But it's main advantage is the containment and the "blank slate".

In the future, I would like to expand on the vision, to include the ability to do deduplication, so that all those zip files with bundled dependencies take up less space. And deduplication done right, will save bandwidth too. Indeed, when you have these "deduplicated zip files", with deduplication done algorithmically, you'll save even MORE space and MORE bandwidth than with a traditional dependency resolving file manager. Take for example debian. Debian uses packages. These packages have dependencies. The dependencies are shared. This saves space over a case where you have a bunch of zip files with unshared dependencies. But when you update a dependency, even if only one line of one file has changed, that dependency must be downloaded completely anew. But with algorithmic de-duplication, updating one line of one file means you only have to download one line of one file anew.

Space savings are there for a similar reason.

Re:Subuser turns Docker containers into normal Linux program... by sandGorgon on Friday February 12, 2016 @10:13AM (#11087696)
It is revolutionary - if you manage to build the ecosystem. Remember that Docker itself was not considered revolutionary ("yawn.. Use lxc").

I would really recommend you look at nix, zeroinstall, click packages and http://0pointer.net/blog/revisiting-how-we-put-together-linu...

Re:Subuser turns Docker containers into normal Linux program... by timthelion on Friday February 12, 2016 @11:38AM (#11088425)
zeroinstall and click are interesting. But zeroinstall doesn't do anything on the security front. Click is rather confusing, I haven't found a webpage for it yet, or any information about how to like install and use a click package on Debian(I think that this is because it's not possible/supported). However, I don't see nix or redhats various efforts(they have annoced a new universal package format every year or two for a decade now) to be very serious. The problem with these efforts is that they always want to impose some opinion on HOW things should be packaged. And I don't think that is useful as a global standard.
Re:Subuser turns Docker containers into normal Linux program... by sandGorgon on Friday February 12, 2016 @01:16PM (#11089178)
you should really kickstarter this project if you are really serious. I think this can really be something if pushed hard. For all its hate, systemd is a one man effort that changed Linux. And so was git.

There are people out there who would love to financially support this if you ask and have a good sense of what you want to do. IMHO your post above (about zeroinstall and click) are jumping the gun.

Would love to see what you come up with.. once the excitement has died down ;)

Re:Subuser turns Docker containers into normal Linux program... by digi_owl on Friday February 12, 2016 @02:03PM (#11089566)
I keep wondering how much traction systemd would have gotten without having a long standing project like udev latched onto it (never mind the consolekit replacement logind).

I'm just waiting for some project to be wholly dependent on the existence of networkd or some such...

Re:Subuser turns Docker containers into normal Linux program... by txutxu on Friday February 12, 2016 @10:27AM (#11087812)
0pointer... after break half of the internet... and need to patch the other half... now releases a pid 2 silently (see release notes for last release), because people was right, because of architecture.

Fortunately still I'm free to use my non market targeted, written by myself, configuration management tool, to pick how i boot my personal systems.

Sorry, will not buy more blog posts from that domain.

Re:Subuser turns Docker containers into normal Linux program... by drdaeman on Friday February 12, 2016 @10:26AM (#11087805)
> to include the ability to do deduplication

But... Won't it be an ugly kludge?

From what I understood, Subuser does introduce the duplication problem just because of the way it works. It didn't exist with traditional package management when done right (because shared components are packaged separately), and it didn't exist with Docker (because it uses layers). If software (any software) essentially adds some issue and then tries to fight it off, it's very likely that things will be ugly in architectural sense.

> But when you update a dependency, even if only one line of one file has changed, that dependency must be downloaded completely anew

Are you aware about debdelta?

Re:Subuser turns Docker containers into normal Linux program... by aidanhs on Friday February 12, 2016 @10:50AM (#11088017)
> and it didn't exist with Docker (because it uses layers)

There's a big spectrum of 'solving the deduplication problem' and Docker is towards the 'when all you have is a sledgehammer' end.

Saying "you can manually arrange Dockerfiles so that they cooperate and share layers" is not solving things in an architectual sense! You essentially have to be using a single custom tool (Dockerfiles) to create your images and then you need to apply thinking power to consider how best to arrange your images (e.g. having a 'base' package.json to install a bunch of things common to many apps, then additional an additional package.json per-app).

It's getting better with 1.10 (layer content is no longer linked to parent layers, so ADDing the same file in different places should reuse the same layer) but it's still pretty imperfect. I created https://github.com/aidanhs/dayer to demonstrate the ability to extract common files from a set of images and use it as a base layer, which is another improvement. Even better would be a pool of data split with a rolling checksum, like the one bup creates - short of domain-specific improvements (e.g. LTO when compiling C), I think this is probably the best architectural thing you can do.

Re:Subuser turns Docker containers into normal Linux program... by drdaeman on Friday February 12, 2016 @11:04AM (#11088145)
Oh, sorry. Yes. I think, I'm with you on this. I didn't meant to say about the quality of this approach. I just meant that Docker has its means to not duplicate by having shared base layers - so we can say the problem [mostly] didn't exist in systems Subuser had started from - but it's completely another matter whenever the approach it takes is good or not.
Re:Subuser turns Docker containers into normal Linux program... by timthelion on Friday February 12, 2016 @10:35AM (#11087880)
No, algorithmic deduplicaton is not a kludge! It is the oposite of a kludge. It is a beautiful way of letting the computer do hard work for you! Rather than trying to deduplicate things by hand (aka, traditional dependency management) you let the computer do it for you :)
Re:Subuser turns Docker containers into normal Linux program... by drdaeman on Friday February 12, 2016 @10:58AM (#11088089)
Oh, no, I guess I wasn't clear on what I mean. Sorry.

Data compression (which deduplication essentially is a subset of) absolutely isn't a kludge. I meant, introducing the duplication (by design) and then inventing some workaround to get back to square one - that's the suspicious part. It adds complexity, when there could be none.

Re:Subuser turns Docker containers into normal Linux program... by timthelion on Friday February 12, 2016 @11:04AM (#11088144)
Square one, being, in your opinion cramming all of the dependencies together in one place? There is a big advantage to immutable dependencies though... Your code NEVER breaks. If the dependencies are immutable, and the architecture stays the same, then your code will run. Of course, you can still use apt-get or another package manager to build your images, so the whole updating of dependencies thing is no worse than where you started. It's just, you have the option of not changing things, and not breaking things as well.
Re:Subuser turns Docker containers into normal Linux program... by drdaeman on Friday February 12, 2016 @11:37AM (#11088412)
I must be misunderstanding something.

Isn't it Subuser that crams all the dependencies together, in one place (image)? So there was that proposal so images are deduplicated, in a sense shared files are automatically detected and stored only once, saving disk space. But then, isn't it that Subuser is completely unaware of any metadata a particular file may have, so it can't really tell the difference between libz and libm, or know that all those binary-different libpng16.so have the 100% compatible ABI and are interchangeable (but not all libpng12.so do)?

From what I understood, Subuser is a package manager (plus permission manager) that doesn't know a thing about what it's packaging - only the large-scale image.

Package managers have all libraries separate, that's the whole point why package managers were invented in the first place. If a package management uses some database, dependencies may lay together in the filesystem, but the're completely separate in package manager's database. If package management system uses filesystem as its database, then packages are separate in that regard, too. There's immutability as well, sometimes enforced, sometimes along the lines of storing `dpkg -l | grep '^ii' | awk '{ print $2 "=" $3 }'` output to save the exact state.

Re:Subuser turns Docker containers into normal Linux program... by txutxu on Friday February 12, 2016 @10:18AM (#11087725)
I will not trust some binary logic that interacts with my data, just because it runs with a dferent numeric uid on my system. That's the point.

People is calling "containers" to operative system images pulled from third party repositories under certain laws.

If we left all that behind, yes, it's better if your firefox cannot read you ssh private key for example. But I don't need docker for that, sorry.

I need a third party blob, when I don't want to embrace the postgresql source code for free and it's manual, and make myself more free tomorrow.

But the... how many millions was it? dollars campaign behind certain technologies (and providers), has more power than any thinking, regarding network effects.

Re:Subuser turns Docker containers into normal Linux program... by timthelion on Friday February 12, 2016 @10:25AM (#11087795)
Running something in subuser is no more of a binary blob than installing it through apt-get. Indeed, it uses apt-get by default! https://github.com/subuser-security/subuser-default-reposito...
Re:Subuser turns Docker containers into normal Linux program... by falsedan on Friday February 12, 2016 @01:11PM (#11089132)
Your needs and point-of-view are pretty specialised. But just because you don't need to use it doesn't mean no one else could benefit.