News for hackers. Stuff that matters.
Re:Microsoft open-sources Graph Engine by jpfed on Thursday February 09, 2017 @10:50AM (#13607728)
I am under the impression that MS has a lot of different parts, and that these parts have different cultures and goals. MS Research and their developer division seem really open-source-friendly, but that may or may not say anything about their OS team.
Re:Microsoft open-sources Graph Engine by Someone1234 on Thursday February 09, 2017 @11:25AM (#13608045)
Their OS team just put a Ubuntu-Linux subsystem into Windows via direct kernel calls. I'd be careful assuming anything about them.
Re:Microsoft open-sources Graph Engine by ygra on Thursday February 09, 2017 @11:29AM (#13608085)
They were also very careful not to touch anything GPL-licensed. The Linux syscalls were all implemented from scratch. So it's effectively a proprietary reimplementation of parts of the Linux kernel. I'm not sure how much that says about open-source plans of Windows itself.
Re:Microsoft open-sources Graph Engine by nxc18 on Thursday February 09, 2017 @12:33PM (#13608851)
That says more about the fact that that implementation pattern is a core part of the NT OS, and the underlying design of the two kernels is so different that just throwing Linux code in the mix would not make sense (in the way that hearing purple doesn't make sense).

WSL is a elimination of the Linux user mode programming environment. The fact that that requires implementing Linux kernel functionality is simply an artifact of the lack of a more modular design (or really any up front design) in Linux.

Re:Microsoft open-sources Graph Engine by poizan42 on Thursday February 09, 2017 @01:09PM (#13609239)
> The fact that that requires implementing Linux kernel functionality is simply an artifact of the lack of a more modular design (or really any up front design) in Linux.

Curiously the recommended syscall mechanism on x86 is by calling __kernel_vsyscall in the vDSO. If everybody did that then you could just make your own loader with your own custom vDSO that could implement the syscalls in userspace. However sometimes (especially statically built) programs still makes syscall directly with int 80h, which are slow to trap in userspace or may not even be possible depending on the os.

Now it would have been fantastic if calling through the vDSO had simply been the only documented way of doing a syscall on x86_64, but the kernel developers at that time decided not to do that, so now on x86_64 the syscall instruction is always used directly, and we can't even trap that on any x86_64 OS because most of the time all that happens is either some unrelated syscall is executed, or an error is returned from the kernel back to the calling process without any traps.

Re:Microsoft open-sources Graph Engine by amluto on Thursday February 09, 2017 @02:03PM (#13609796)
vDSO maintainer here.

> Curiously the recommended syscall mechanism on x86 is by calling __kernel_vsyscall in the vDSO.

There are times when this doesn't work. Syscall resumption and cancellation come to mind. Also, __kernel_vsyscall is a hack to make fast syscalls work on the awful 32-bit x86 architecture, not a nice feature.

> it would have been fantastic if calling through the vDSO had simply been the only documented way of doing a syscall on x86_64

There is no __kernel_vsyscall or similar feature on x86_64.

> we can't even trap that on any x86_64 OS

You can on Linux using seccomp.

Re:Microsoft open-sources Graph Engine by poizan42 on Friday February 10, 2017 @06:54AM (#13614753)
> There are times when this doesn't work. Syscall resumption and cancellation come to mind. Also, __kernel_vsyscall is a hack to make fast syscalls work on the awful 32-bit x86 architecture, not a nice feature.

It's a pretty nice feature in the context of being able to make compatibility layers on other OS'ses in userspace which was the discussion here. Or would be if it was always used. Why doesn't it work for syscall resumption and cancellation?

> There is no __kernel_vsyscall or similar feature on x86_64.

Yes that was exactly what I was complaining about.

> You can on Linux using seccomp.

Yes but why would I want to make a linux compatibility layer on linux?

Re:Microsoft open-sources Graph Engine by amluto on Friday February 10, 2017 @06:18PM (#13619776)
> It's a pretty nice feature in the context of being able to make compatibility layers on other OS'ses in userspace which was the discussion here. Or would be if it was always used. Why doesn't it work for syscall resumption and cancellation?

For resumption, a signal that interrupts a resumable syscall points RIP to an explicit int 80 instruction in the vDSO. This behavior would be a bit unfriendly to emulate.

For cancellation, the only good implementation of cancellation that I'm aware of (musl's) relies on syscalls being an actual atomic instruction so that a signal handler can tell whether a syscall actually happened. __kernel_vsyscall is an opaque function and can't be used like this.

> Yes but why would I want to make a linux compatibility layer on linux?

For sandboxing? For experimentation? Or how about to make a compatibility layer emulating something else that runs on Linux?

Re:Microsoft open-sources Graph Engine by eganist on Thursday February 09, 2017 @11:59AM (#13608457)
That sounds more like an edict from MS Legal more than anything related to the culture of the core OS team.
Re:Microsoft open-sources Graph Engine by pcnix on Thursday February 09, 2017 @11:43AM (#13608263)
I was surprised at how well it was implemented. It's a very comprehensive system, and their roadmap for the future is promising too.
Re:Microsoft open-sources Graph Engine by yulaow on Thursday February 09, 2017 @11:48AM (#13608324)
Can I ask you where I can find their roadmap about it? Very interested recently
Re:Microsoft open-sources Graph Engine by Analemma_ on Thursday February 09, 2017 @02:52PM (#13610282)
I don't know if there's a single document somewhere with an explicit roadmap. Their GitHub page (https://github.com/Microsoft/BashOnWindows) includes links to multiple sources that could collectively be considered a sort of roadmap, including the Issue tracker, UserVoice page, team blog, and discussion forums. I've also asked the devs questions on Twitter a couple times (like @richturn_ms) and usually gotten fast responses.
Re:Microsoft open-sources Graph Engine by bitcrazed on Monday February 13, 2017 @12:47PM (#13637192)
You called? ;)

Also take a look at our Command-line blog (https://blogs.msdn.microsoft.com/commandline) for LOTS more about Console & Bash/WSL, including the page where I pull together our various learning resources into one place: https://blogs.msdn.microsoft.com/commandline/learn-about-bas...

Regarding roadmap, to be honest, we don't really have one, but we do have a primary mission which we're laser-focused on: Make Windows Subsystem for Linux able to run the vast majority of tools developers need to get their work done.

We've taken a somewhat unusual, but HIGHLY valuable approach of building features based almost entirely on feedback from the community. We prioritize syscall implementation based on the frequency with which we see failures due to missing, incomplete, or incorrect syscalls. This way, we focus on delivering the most bang for the buck.

This said, now that our syscall coverage is starting to edge towards covering the majority of mainstream dev tools, we're starting to look at more strategic, features including improving disk perf & support for networking, device, etc.

We'll be publishing more details in a few weeks on our blog above (and, of course via https://twitter.com/richturn_ms) on where we're going to invest effort beyond Creators Update. Look forward to hearing from y'all ;)

Re:Microsoft open-sources Graph Engine by whatever_dude on Thursday February 09, 2017 @01:43PM (#13609596)
I'm interested in this roadmap as well. I currently follow their bash changelog blog, and check that user vote site, but I haven't seen anything like a more concise future plan.