Re:Three New Open Source Container Utilities
by Illniyar on Thursday July 06, 2017 @02:22AM
(#14708508)
Open source or not, I'll do my best tp avoid any product by oracle. They have shown to value short-term profit above both community and even PR, which makes using their products an uneasy gamble for me.
Re:Three New Open Source Container Utilities
by jsmthrowaway on Thursday July 06, 2017 @02:34AM
(#14708547)
They're all Apache 2, which is about as permissively decoupled from the organization as it gets. Oracle could disappear tomorrow, delete all the repos, or begin charging $5,000/seat for this software, and nothing would happen. You could, literally, fork all three, rename them, and run off on your own projects and have a merry 'ole time.
In fact, everyone going after Oracle in this thread should do exactly that right now.
Re:Three New Open Source Container Utilities
by Illniyar on Thursday July 06, 2017 @02:48AM
(#14708602)
You are right in theory, however it's very rare for abandoned or mishandled OSS projects to gain new life by forking. Most of the knowledge, expertise and incentives are tightly coupled with the original creators.
Also, open source license doesn't cover patents, so they could decide to sue for patent infringement, which I don't thunk is likely, but if there ever was a company that might try something like that it's oracle.
The patent assignment and retaliation clause, in Apache 2.0 §3, is one of the key reasons to use it. Your hypothetical would not happen.
> however it's very rare for abandoned or mishandled OSS projects to gain new life by forking.
Given the counterexamples that have sprung to life from Oracle's behavior in the past (MariaDB, Jenkins), I find this a very odd observation in this thread.
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @03:27AM
(#14708720)
The Apache-2.0 patent clause inspired the GPLv3 patent clause (previously Apache-2.0 was incompatible with GPLv2 because GPLv2 didn't have a patent clause).
Re:Three New Open Source Container Utilities
by steveklabnik on Thursday July 06, 2017 @05:25PM
(#14714091)
This is why MIT/Apache2 dual licensed is popular; Apache2 contains a patent clause, but isn't GPL compatible, so you can use the Apache2 variant if you don't use anything GPL, but use the MIT variant if you have to have compatibility.
Re:Three New Open Source Container Utilities
by pjmlp on Thursday July 06, 2017 @02:38AM
(#14708557)
I have been using Oracle products since around 1996 and will keep using them.
MS SQL Server is the only one that can compete with PL/SQL, the feature lists, and the GUI tooling provided out of the box for database administration, report generation and data modeling.
Java, they have made quite a few improvements to the language and runtime, some of them even unthinkable under Sun's stewardship, namely adding Graal to the official JDK and adding AOT support (most commercial JDK have it since early days of Java).
Also their roadmap for value types, GPGPU, fully replacing Hotspot with Graal and eventually having OpenJDK mostly implemented in Java, after those changes are done.
They created SPARC M7 with Silicon Secured Memory, as yet another band-aid against C security exploits.
So while the company might be heavy profit oriented, it is no different than any other corporation that likes money.
Re:Three New Open Source Container Utilities
by gtirloni on Thursday July 06, 2017 @08:06AM
(#14709710)
One can argue VBox has been on life support since Oracle acquired it from Sun. I'm puzzled about why they haven't shut it down too.. maybe some profitable support contracts? Internal usage?
Re:Three New Open Source Container Utilities
by pjmlp on Thursday July 06, 2017 @06:40AM
(#14709365)
If you look at my comment history you will see that as big Xerox PARC and ETHZ fan, I am not a big supporter of UNIX in general.
Last time I touched Solaris in production was in 2002.
Yeah I did play with the OpenSolaris and Solaris 10/11 images, but really didn't saw any value on actually using it.
The fact is that *BSD and GNU/Linux variants killed the commercial UNIXes in the server room, the variants being sold are mostly tied to maintenance contracts.
Even Sun wouldn't be able to keep their Solaris business going afloat for much long.
How much businesses do bother to actually buy new hardware to run Aix or HP-UX on it?
Re:Three New Open Source Container Utilities
by gtirloni on Thursday July 06, 2017 @08:13AM
(#14709755)
Sun had a chance with OpenSolaris and they blew it with terrible community management. The fact that they wanted OpenSolaris to be Solaris Next with minimum to no changes posed a serious conflict with community contributions. If they had followed the Fedora/RHEL strategy things would have been different but there always that mistrust of external contributions (e.g. nobody outside Sun/Oracle could build the thing properly, let alone the binary blobs issue).
We had hundreds of OpenSolaris servers and nobody could say that in 2008-2010 they weren't the best we had for a NAS server. Linux and BSD didn't come close to having anything resembling SMF/ZFS/etc... nowadays it's a different story, I'd pick FreeBSD any day for a NAS appliance.
Re:Three New Open Source Container Utilities
by gtirloni on Thursday July 06, 2017 @09:06AM
(#14710047)
I haven't followed Illumos lately. At the time we evaluated it and its community and it didn't seem worth it (keep in mind we were deeply depressed about anything Solaris). Maybe it's thriving today? I wouldn't know. I started using FreeBSD around 2000-2001 and so there's a personal bias there too.
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @10:43PM
(#14715636)
From what I've heard, illumos has quite a strong community now. The events surrounding OpenSolaris caused an exodus from Oracle, and everyone landed either in illumos or in FreeBSD.
These days development of ZFS is done as OpenZFS with members of FreeBSD, illumos and ZoL. DTrace development mostly happens in illumos.
All of the above comes from second-hand experience based on what I've heard from the folks at Joyent (now part of Samsung) such as bcantrill. I have only played with FreeBSD and illumos and read through some of the code.
Re:Three New Open Source Container Utilities
by joseluisq on Thursday July 06, 2017 @01:43AM
(#14708383)
I transcribe the following:
"Go is a poor choice of language for a container runtime," Abrams wrote in a blog post." Go is a great language, but for small system utilities that need tight control over threads and make a high volume of syscalls, there are better options."
Re:Three New Open Source Container Utilities
by solidsnack9000 on Thursday July 06, 2017 @02:08AM
(#14708462)
Abrams is the author of the Go Netlink library, and participates in an interesting conversation here: https://news.ycombinator.com/item?id=14470231 (as `vishvananda`, it's the top comment).
Many of the things the commenters discuss there are intricate. It's a lot of people who've worked on container runtimes and the issues relate to the interaction of Go with POSIX process and thread semantics. Go is not really intended to have transparent, finely controlled integration with C systems stuff...while Rust is.
Re:Three New Open Source Container Utilities
by jitl on Thursday July 06, 2017 @02:41AM
(#14708570)
I recently came up against this when deciding on a language to implement a POSIX-api backed utility.
Go does not have the fork(2) POSIX API. This makes it much more difficult to spawn a process with precise POSIX semantics.
Another issue is that when a goroutine (a fiber running in an OS thread) makes a syscall that blocks, that goroutine might be un-scheduled and then later resumed on a different OS thread. If your syscall is one that operates on the call-in OS thread, this could cause poor results. I think there are some mitigation strategies, but these two issues make syscall programming frustrating in Go.
Re:Three New Open Source Container Utilities
by Veratyr on Thursday July 06, 2017 @03:08AM
(#14708666)
How is this relevant to the question raised by the person you're replying to? You've raised an entirely different concern and ignored the question. Why did you post this as a reply rather than a top level comment?
Re:Three New Open Source Container Utilities
by vishvananda on Thursday July 06, 2017 @11:48AM
(#14711261)
Author here. This hit hacker news in the middle of the night my time. I think the other replies already answered your question but it is worth mentioning that my point could have been worded better. Note that go works very well for other parts of the container system: docker client, docker daemon, containerd, kubernetes, etc. It is just the low-level namespace handling it has trouble with.
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @03:11AM
(#14708673)
Absolutely. There was a recent thread about how Linux namespaces and Go really don't like each other. There are many more issues with Go. Maybe I should write a blog post about it.
Re:Three New Open Source Container Utilities
by vishvananda on Thursday July 06, 2017 @11:51AM
(#14711276)
Author here. For more discussion on init handling you may find my article on pid namespaces[1] enlightening. In terms of differences from runc, there aren't many. The goals for railcar are threefold:
1: provide an alternative implementation of the oci-runtime so that the spec doesn't become too locked to a single implementation.
2: provide an implementation in a single language without some of the "baggage" of the existing implentations so that it is easy and fun to hack on.
3: experiment with new ideas to inform the future of the oci-runtime spec.
Re:Three New Open Source Container Utilities
by rndmio on Thursday July 06, 2017 @03:09AM
(#14708667)
One thing it means is no zombies in the process table. When you're running something like squid in a container and you HUP it to reload the config (volume mounted) without something to reap the old process you're going to have zombies.
Re:Three New Open Source Container Utilities
by insulanian on Friday July 07, 2017 @04:46AM
(#14717003)
Having read the comments here about Go not being suited for the low level stuff, I really wonder why did the designers of Go think it would be THE replacement for C/C++?
Re:Three New Open Source Container Utilities
by conradk on Thursday July 06, 2017 @02:38AM
(#14708558)
Probably because Smith (1 of the 3 tools) is written in Go and because Oracle wouldn't use Rust just for the sake of using Rust but because it was suited to the task at hand.
edit: you might be interested in their article on "Building a Container Runtime in Rust" [1]
Re:Three New Open Source Container Utilities
by pjmlp on Thursday July 06, 2017 @05:09AM
(#14709033)
With that we have now Microsoft and Oracle adopting Rust based tools.
Even if Rust fails to gain wide adoption, their ideas already tainted (in a very positive way) future work on the design of Swift, Pony, D and C++ lifetime static analyzers.
Also on Go side, Microsoft being a Docker contributor regarding its support on Azure and VS Code tooling.
Which I find very positive for both communities as a language geek.
Re:Three New Open Source Container Utilities
by jondubois on Thursday July 06, 2017 @02:33AM
(#14708545)
What sucks about open source these days is that companies keep reinventing the wheel with their own branding instead of helping improve the wheel that already exists.
I really don't buy it that Go is a poor choice but that Rust isn't. If they had used C/C++, then the argument would sound more compelling.
I think that Oracle has forever tarnished its open source credentials. These days Oracle is synonymous with vendor lock-in which is the antithesis of open source. I think they would have had more success if they had done the announcement anonymously.
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @03:22AM
(#14708700)
> What sucks about open source these days is that companies keep reinventing the wheel with their own branding instead of helping improve the wheel that already exists.
While that might be true in other contexts, the tools Oracle has been working on all work with the OCI standards. I actually completely disagree with creating a monoculture around a standard (that's how you get OpenSSL), and the whole reason why I contribute to the OCI standard is specifically so companies are free to create their own thing as long as users have the freedom to change between implementations (or write their own).
The situation right now is better than it was 3 years ago before the OCI existed and everyone was trying to make Docker fit their own usecases and needs.
> I really don't buy it that Go is a poor choice but that Rust isn't.
Take it from me [I'm a maintainer of runc], they're right. I would actually put it in far stronger terms than TFA, but Go has many problems that Rust does not. On the other hand, C/C++ have many problems that Go does not (which is why they went with Rust -- and I applaud them for it).
> These days Oracle is synonymous with vendor lock-in which is the antithesis of open source.
OCI is an open standard, anyone can contribute. Literally the precise opposite of vendor lock-in. I don't like Oracle either, but bashing them when they do something good is not helping.
Re:Three New Open Source Container Utilities
by pjmlp on Thursday July 06, 2017 @02:42AM
(#14708576)
> I really don't buy it that Go is a poor choice but that Rust isn't. If they had used C/C++, then the argument would sound more compelling.
It all boils down to Go's runtime implementation and their decisions regarding signal and threading handling.
It is because of these little things that Go isn't a fully systems language capable of replacing C and C++ for close to the hardware tasks.
For normal user space stuff sure, for kernel or low level tricks, if it requires more hand-written Assembly than C or C++ do, then it is a lost game trying to convince people to use it.
Re:Three New Open Source Container Utilities
by jsmthrowaway on Thursday July 06, 2017 @02:40AM
(#14708563)
> What sucks about open source these days is that companies keep reinventing the wheel with their own branding instead of helping improve the wheel that already exists.
Competition is a good thing overall, particularly when it is a corporation with a lot of resources putting some of them to creating an alternative. The alternatives to Docker have made them improve drastically. Uncharitably, this observation sounds like it's colored by your opinion of Oracle more than the structure itself: what if Google put out these three tools? I also strongly condemn a world where people are discouraged from exploring new approaches because something else, often something that a commenter happens to like, exists. Trust the market to speak.
> I really don't buy it that Go is a poor choice but that Rust isn't.
They're not selling it, WeaveWorks is, and the specific (legitimate) issue has been extensively debugged on go-nuts and elsewhere:
It's dark corners of hardware and low-level APIs where Go suffers. Platform-heavy concepts in C often translate more readily to Rust than Go, but that's not a knock of any language -- just different targets. I've been down the syscall/C ABI/etc path in Go, and most of that stuff (oddly, given that the stdlib exercises it) feels bolted on to the language as an afterthought.
Rust, OTOH, intentionally sailed the "C But Better" tack. Interop between C and Rust is quite pleasant, and that is the gateway drug to all sorts of hardware-heavy tricks that really exercise a language. The same tricks in Go tend to surface weird stuff like the aforementioned issue.
That being said, generalizing to an indictment of Go as a language for this use case is flawed. That first sentence under Railcar could use some work.
Re:Three New Open Source Container Utilities
by joseluisq on Thursday July 06, 2017 @02:45AM
(#14708586)
It's true, but to affirm that "Go is a poor choice of language for a containers" it would be equivalent to say that Docker did a poor language choice... ?
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @03:24AM
(#14708708)
Docker didn't have much of a choice. At the time they had two options: C/C++ or Go (Rust wasn't usable at). They went with the right choice at the time, Go. Unfortunately nobody really knew about all of the pitfalls of Go (there are many upsides, but writing a container runtime in Go is definitely not one of them).
Right now, I would definitely recommend people write things like this in Rust. In fact I was planning on working on something similar. :P
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @05:24AM
(#14709094)
I'm not a Rust expert (yet) but I probably should write something up because this topic always comes up when people like me start complaining about Go.
Re:Three New Open Source Container Utilities
by vishvananda on Thursday July 06, 2017 @11:56AM
(#14711315)
Author here. As I mentioned in another comment. I should have clarified that its shortcomings are specific to the container runtime (i.e. the part that creates and enters the namespaces). Go is a great choice for the command line utilities and daemons. I am a big fan of go in general, and even the container builder smith is written in go.
Re:Three New Open Source Container Utilities
by jsmthrowaway on Thursday July 06, 2017 @02:49AM
(#14708605)
Don't generalize from this post. That's a trap. Go's runtime model doesn't interact well with stateful namespace syscalls in the Linux kernel, and it's hard to fix. That's all that can be concluded.
Re:Three New Open Source Container Utilities
by cyphar on Thursday July 06, 2017 @03:26AM
(#14708714)
We've had production issues where Go's memory model will cause programs such as dockerd to crash due to overzealous memory over-commitment (Go doesn't actually free memory they use MADV_DONTNEED). There are definitely problems with Go that go outside of "stateful namespace syscalls".
Re:Three New Open Source Container Utilities
by pjmlp on Thursday July 06, 2017 @05:18AM
(#14709068)
Go could be in the same space as Oberon was for systems programming (real OS stuff and bare metal coding), but for that to happen they would need to improve the runtime and unsafe package for those scenarios.
In fact, everyone going after Oracle in this thread should do exactly that right now.
Also, open source license doesn't cover patents, so they could decide to sue for patent infringement, which I don't thunk is likely, but if there ever was a company that might try something like that it's oracle.
http://en.swpat.org/wiki/Patent_clauses_in_software_licences...
The patent assignment and retaliation clause, in Apache 2.0 §3, is one of the key reasons to use it. Your hypothetical would not happen.
> however it's very rare for abandoned or mishandled OSS projects to gain new life by forking.
Given the counterexamples that have sprung to life from Oracle's behavior in the past (MariaDB, Jenkins), I find this a very odd observation in this thread.
MS SQL Server is the only one that can compete with PL/SQL, the feature lists, and the GUI tooling provided out of the box for database administration, report generation and data modeling.
Java, they have made quite a few improvements to the language and runtime, some of them even unthinkable under Sun's stewardship, namely adding Graal to the official JDK and adding AOT support (most commercial JDK have it since early days of Java).
Also their roadmap for value types, GPGPU, fully replacing Hotspot with Graal and eventually having OpenJDK mostly implemented in Java, after those changes are done.
They created SPARC M7 with Silicon Secured Memory, as yet another band-aid against C security exploits.
So while the company might be heavy profit oriented, it is no different than any other corporation that likes money.
It is the worst company that we in the open source community know of.
Everything they touch dies. If it does not die, it probably should, probably even for technical reasons.
Hudson (now Jenkins)
JavaEE 8 (delayed, uncertainity around at least one JCP. Really poor communication.)
- Java3D
- JOGL
- Java Desktop Integration apis
- Java packaging for desktop apps (Oracle eventually did it with the JavaFX reboot)
- Requiring devs to master the "Filthy Rich Clients" book in order to create cool Swing interfaces
- Making a mainstream product out of Sun SPOT before Arduinos and RaspberryPIs were a thing
- Ensuring J2ME fragmentation wasn't the way it turned out to be
- Supporting AOT compilation to native code, while all commercial vendors selling JDKs were doing it since the early days.
- Creating a GNOME based desktop named Java Desktop System
Sun wasn't all flowers and hugs regarding Java development.
Last time I touched Solaris in production was in 2002.
Yeah I did play with the OpenSolaris and Solaris 10/11 images, but really didn't saw any value on actually using it.
The fact is that *BSD and GNU/Linux variants killed the commercial UNIXes in the server room, the variants being sold are mostly tied to maintenance contracts.
Even Sun wouldn't be able to keep their Solaris business going afloat for much long.
How much businesses do bother to actually buy new hardware to run Aix or HP-UX on it?
We had hundreds of OpenSolaris servers and nobody could say that in 2008-2010 they weren't the best we had for a NAS server. Linux and BSD didn't come close to having anything resembling SMF/ZFS/etc... nowadays it's a different story, I'd pick FreeBSD any day for a NAS appliance.
These days development of ZFS is done as OpenZFS with members of FreeBSD, illumos and ZoL. DTrace development mostly happens in illumos.
All of the above comes from second-hand experience based on what I've heard from the folks at Joyent (now part of Samsung) such as bcantrill. I have only played with FreeBSD and illumos and read through some of the code.
Is this true ?
Many of the things the commenters discuss there are intricate. It's a lot of people who've worked on container runtimes and the issues relate to the interaction of Go with POSIX process and thread semantics. Go is not really intended to have transparent, finely controlled integration with C systems stuff...while Rust is.
Go does not have the fork(2) POSIX API. This makes it much more difficult to spawn a process with precise POSIX semantics.
Another issue is that when a goroutine (a fiber running in an OS thread) makes a syscall that blocks, that goroutine might be un-scheduled and then later resumed on a different OS thread. If your syscall is one that operates on the call-in OS thread, this could cause poor results. I think there are some mitigation strategies, but these two issues make syscall programming frustrating in Go.
It might be successful but if the name Oracle shows up I am scared.
https://news.ycombinator.com/item?id=14479284
> railcar always runs an init process separately from the container process
What does that mean in terms of day to day operations/debugging ? What are the benefits ?
1: provide an alternative implementation of the oci-runtime so that the spec doesn't become too locked to a single implementation.
2: provide an implementation in a single language without some of the "baggage" of the existing implentations so that it is easy and fun to hack on.
3: experiment with new ideas to inform the future of the oci-runtime spec.
[1] https://hackernoon.com/the-curious-case-of-pid-namespaces-1c...
edit: you might be interested in their article on "Building a Container Runtime in Rust" [1]
[1] https://blogs.oracle.com/developers/building-a-container-run...
Even if Rust fails to gain wide adoption, their ideas already tainted (in a very positive way) future work on the design of Swift, Pony, D and C++ lifetime static analyzers.
Also on Go side, Microsoft being a Docker contributor regarding its support on Azure and VS Code tooling.
Which I find very positive for both communities as a language geek.
I really don't buy it that Go is a poor choice but that Rust isn't. If they had used C/C++, then the argument would sound more compelling.
I think that Oracle has forever tarnished its open source credentials. These days Oracle is synonymous with vendor lock-in which is the antithesis of open source. I think they would have had more success if they had done the announcement anonymously.
While that might be true in other contexts, the tools Oracle has been working on all work with the OCI standards. I actually completely disagree with creating a monoculture around a standard (that's how you get OpenSSL), and the whole reason why I contribute to the OCI standard is specifically so companies are free to create their own thing as long as users have the freedom to change between implementations (or write their own).
The situation right now is better than it was 3 years ago before the OCI existed and everyone was trying to make Docker fit their own usecases and needs.
> I really don't buy it that Go is a poor choice but that Rust isn't.
Take it from me [I'm a maintainer of runc], they're right. I would actually put it in far stronger terms than TFA, but Go has many problems that Rust does not. On the other hand, C/C++ have many problems that Go does not (which is why they went with Rust -- and I applaud them for it).
> These days Oracle is synonymous with vendor lock-in which is the antithesis of open source.
OCI is an open standard, anyone can contribute. Literally the precise opposite of vendor lock-in. I don't like Oracle either, but bashing them when they do something good is not helping.
It all boils down to Go's runtime implementation and their decisions regarding signal and threading handling.
It is because of these little things that Go isn't a fully systems language capable of replacing C and C++ for close to the hardware tasks.
For normal user space stuff sure, for kernel or low level tricks, if it requires more hand-written Assembly than C or C++ do, then it is a lost game trying to convince people to use it.
EDIT: Removed too many "level" typos
It seems like their container builder, Smith, creates OCI [1] images, an open format that can be used from other tools such as runc or rkt.
[1] https://github.com/opencontainers/image-spec
Competition is a good thing overall, particularly when it is a corporation with a lot of resources putting some of them to creating an alternative. The alternatives to Docker have made them improve drastically. Uncharitably, this observation sounds like it's colored by your opinion of Oracle more than the structure itself: what if Google put out these three tools? I also strongly condemn a world where people are discouraged from exploring new approaches because something else, often something that a commenter happens to like, exists. Trust the market to speak.
> I really don't buy it that Go is a poor choice but that Rust isn't.
They're not selling it, WeaveWorks is, and the specific (legitimate) issue has been extensively debugged on go-nuts and elsewhere:
https://www.weave.works/blog/linux-namespaces-and-go-don-t-m...
It's dark corners of hardware and low-level APIs where Go suffers. Platform-heavy concepts in C often translate more readily to Rust than Go, but that's not a knock of any language -- just different targets. I've been down the syscall/C ABI/etc path in Go, and most of that stuff (oddly, given that the stdlib exercises it) feels bolted on to the language as an afterthought.
Rust, OTOH, intentionally sailed the "C But Better" tack. Interop between C and Rust is quite pleasant, and that is the gateway drug to all sorts of hardware-heavy tricks that really exercise a language. The same tricks in Go tend to surface weird stuff like the aforementioned issue.
That being said, generalizing to an indictment of Go as a language for this use case is flawed. That first sentence under Railcar could use some work.
Right now, I would definitely recommend people write things like this in Rust. In fact I was planning on working on something similar. :P