Pulseaudio was Lennart's previous project. It broke everything in linux sound for a while, everybody moaned and hated it and said it was the worst thing since the crucifixion of Christ.

Yet, name one problem you had with sound on linux in the past year? There are very few. Pulseaudio now just works(tm) and is a unseed, unheard of part of the plumbing.

If you remember what is was like messing with ALSA and (shudders) OSS before pulseaudio came along you will agree that the current state of affairs is a million miles better. It used to be really difficult to get more than one application to be able to play sound at a time. I remember compiling sound drivers from source just to get them working. Configuring ALSA config files to get surround sound working was practically a black art. Creating manual scripts that unmute the sound card on every boot because the driver didn't initialize it properly.

With pulseaudio, I never have to worry about any of that and configuring surround sound takes me two clicks of the mouse.

Lennart did a fantastic job with pulseaudio, he took on a dirty problem that nobody else dared to touch and went through years of criticism to produce a really high quality solution that solved the linux audio problem so well that you don't hear complaints about it anymore.

In light of that, I trust him to do a good job with systemd. It'll be a couple of years of everyone moaning and bitching and whining about it, then one day it will have become a seamless part of the plumbing, everyone will take it for granted and wonder how they ever managed fighting with shell scripts and fragmented init systems before systemd came along.

It's ironic that Lennart Poettering is probably the most abused developer in the entire OSS ecosystem, yet he is one of the people contributing most to it. For our sake, I'm glad he has such a thick skin. If I was him I'd have quit this game long ago.

That's just it. Linux sound worked fine for me before Pulseaudio, and FreeBSD sound has always worked perfectly fine for me. In fact, FreeBSD solved sound mixing sooner via /dev/pcm virtualization (while Linux chose to create the Linux-only ALSA instead), and has always had lower observed latency.

Pulseaudio screwed up my audio so badly that for a year I was running the closed source OSSv4 binaries and manually recompiling all the audio libraries to use OSS instead of ALSA/Pulse.

It is not fantastic to push horribly broken code onto the entire Linux userbase while others frantically jump in to help patch and fix the trainwreck.

And we're doing the same thing again with systemd. Instead of having a few years where users can choose between systemd, sysvinit, openrc or upstart, while all of the major bugs are worked out, we're being forced immediately from sysvinit (Wheezy) to systemd (Jessie). I was on Lennart's treadmill with Pulse, I'm not getting on it again with systemd.

Now PulseAudio was released into the wild too soon by too many distros BUT it has fundamentally fixed what was HORRIBLE in Linux. (Previously a Sound Engineer and Record Studio owner)

BUT I would say that Systemd is extremely stable and not broken. What people are complaining about is the philosophy aspect.

To be fair, I didn't say I never had Linux audio issues prior to Pulseaudio (whereas I did say that about FreeBSD.)

Back in '98, my SB16 ISA card would only output sound at 8-bit monaural under mikmod, and I could only play CD-audio with that passthrough cable between the CD-ROM drive and the sound card. Once I was able to get sound working well enough, the only way I was able to play MIDIs was through Timidity and Soundfont emulation. And until ALSA, there was obviously pain whenever two things would want to play sound at the same time. This of course was due to the OSSv3 author changing the license before introducing his own audio mixing, and all of those awful sound server daemons (esd et al) never really worked, since there were multiple daemons and each application wanted different daemons or just wanted to stab right at the OSSv3 ioctl's.

But once ALSA was established and working, yes. Audio under Linux at that point worked just fine for me. Pulseaudio was a solution looking for a problem.

> (Previously a Sound Engineer and Record Studio owner)

I won't claim to be either of these. I like to listen to music while I write code, I'll occasionally watch some movies or play some games, and I want Pidgin to make a chime when someone sends me a message.

In particular, I'm very sensitive to latency in gaming (emulation), but that's about the extent of what I need speaker sound output for.

> What people are complaining about is the philosophy aspect.

To me, the worst part is the backroom politics, the complete disregard for portability, and the lock-in effects of consuming other daemons and services, and making software dependent upon it.

However, I do also object to the design itself, as well as to the developers responsible for working on the project, and the attitude of disdain they present to the community at large.

The issue was ALSA was HUGE latency to use for anything in recording was just not doable! I had to buy a closed source solution under Windows. Today I could easily do it in Linux.

    /* A */ sample = (sample_a >> 1) + (sample_b >> 1);  //lowers volume of A and B by 50%
    /* B */ sample = max(-32768, min(+32767, sample_a + sample_b));  //prone to clamping

Playing this up as a bogeyman for not being in user-space is FUD, especially when video card drivers also run in kernel space, and are literally thousands upon thousands of times more complex and error-prone. And now the big push is to have kernel mode setting for video cards (even FreeBSD is doing this), which I believe to be a terrible direction to go in.

I have never in my entire life seen a system crash due to audio mixing, but I've personally experienced plenty of video card drivers causing kernel page faults.

If people were even remotely serious about the protection of kernel space (and I certainly wish they were), Minix would be more than a footnote in history. Neither Linux nor the BSDs make serious efforts at microkernel designs. Not even passive attempts to run non-critical device drivers under ring 1. Personally, I'm really rooting for Minix 3 and hope that it takes off more now that it's gained binary compatibility with NetBSD.

I just find it amusing how a monolithic design of doing all audio stuff in the kernel is held up by some as an example of reliability and as superior to a more modular design that is more in line with the UNIX philosophy.

About the KMS however, I've heard that DisplayPort link training has latency requirements that are difficult to meet in anything but a kernel interrupt handler... a quick duckduckgo search finds a short note about that on: http://fedoraproject.org/wiki/Features/RadeonDisplayPort

Also X servers have traditionally needed direct PCI bus access to get the hardware initialized, which means that a buggy X server can hang your PCI bus so the driver running in user space likely doesn't increase reliability in practice.

It's an interesting question to what extent the limited success of microkernel based UNIX implementations is to historical accidents and network effects, and to what extent due to actual technical limitations and additional complexity of a microkernel architecture.

Okay, my apologies as well then. It was hard to get a read from just that one sentence with the word kernel emphasized.

> (Do audio devices support floating point formats nowadays?)

Natively, no. You can be lazy and do it anyway in software mixing though.

> I just find it amusing how a monolithic design of doing all audio stuff in the kernel is held up by some as an example of reliability and as superior to a more modular design that is more in line with the UNIX philosophy.

Certainly, it would be ideal if everything non-critical were in user space. But audio in the kernel is probably at the very bottom of the list. Audio mixing is maybe 0.0001% of the kernel code, and is some of the safest, simplest arithmetic code imaginable. It's worrying about the one ant you saw on the counter when your entire house is infested with termites.

> About the KMS however, I've heard that DisplayPort link training has latency requirements that are difficult to meet in anything but a kernel interrupt handler

I don't know if that's true or not, but I am running a DisplayPort monitor (ZR30w) now without KMS, and it works fine. Obviously the video driver is still running in kernel mode, but at least it's a module outside of the kernel itself that runs after my system is booted.

What I'd really like to see is distros and vendors instead relying on UEFI GOP for boot-time mode setting.

> Also X servers have traditionally needed direct PCI bus access to get the hardware initialized

Well, compare it to audio. Eventually even a userland mixer will have to send the samples through some sort of hardware interface. But if your goal is stability, then it would be ideal to get as much code out of the kernel as possible.

> and to what extent due to actual technical limitations and additional complexity of a microkernel architecture.

Certainly nothing is ever perfect. There are so many potential problems with computers. Cosmic rays can flip bits in your RAM if you don't shell out an extra $500 for the premium CPU, mainboard and ECC RAM. Strong enough power surges (lightning) can burn through and destroy absolutely any running computing equipment. Hardware can literally fail and take down your system. Things can overheat, there can be design flaws in the silicon itself, etc.

So I look at it like OpenBSD looks at security. You want to stack all the protections you can. Mirror your drives, use ECC RAM, don't run anything in kernel space you don't have to, try and build as much redundancy and safety as you can into the system. It won't be perfect, but every bit will help increase uptime.

...

So again, sure, audio should preferably be in user space. Just, it's many thousands of times worse that video isn't even trying to do this, and is in fact going in the opposite direction to become more tightly coupled with the kernel.

However, with one small caveat: servers don't generally have sound cards so the impact of this was relatively low. There aren't that many desktop Linux users out there. I mean I'm a Unix guy at heart and I'm typing this on a Windows laptop. I've never used Linux on the desktop and probably never will.

Now servers do have init processes and we don't really want to spend the next 3-4 years being guinea pigs. I'm quite happy for the vendors to do this behind the scenes or offer it as an alternative but we've got an RHEL+CentOS release with systemd in it already and a Debian with systemd in it just around the corner. A pulseaudio situation, even for 6 months, will result in no small amount of chaos.

I do indeed remember times before even ALSA when you had to pay OSS for drivers for your turtle beach card etc. But that's in the distant past, not right now and of little relevance. Windows was fine on the desktop then as well and the sound worked fine out of the box.

Then you're not really a Unix guy at heart.

At home, all I run is Linux, including the laptop my non-geeky wife uses.

For me it was a hard choice. I knew she would object because it would be "different" and she's not really interested in learning a gazillion different computing-systems, but on the flip side it meant it was simpler, quicker and less work for me to maintain the computers at home.

Once setup things just work, and ensuring everything (including flash and other vulnerability vectors) is up to date is one apt-get upgrade away.

I'm not sure that's a great vote of confidence for the road ahead of systemd (given that systemd presumably has a bit more to it than PulseAudio). To quote the article, "I do honestly believe this will end up being the start of a rocky period for Linux".

Ubuntu 14.04 will keep me happy for ~5 years, then I can take another look at what the current state of things are.

It is only since 14.04 that you have a small chance that opting to use pulseaudio is the better choice.

Just trying to get Skype working (which uses pulseaudio) cost me 2 hours last week, which is not at all nice when you have a call starting in 5 minutes.

Pulseaudio still won't detect the headphone jack on my old intel board, and Skype on my newer machines on Linux will routinely fuckup playback.

One also wonders how much of the PA cleanup was handled by people that weren't Lennart.

In particular for problems of getting Youtube (or any browser audio) to work while other apps use JACK directly.

Although on a recent new install it seemed to work without them as well.

One problem is I need to start/stop the qJackCtl thing every time my laptop comes back from sleep, to get sound working again. There must be a way to automate (or, preferably, fix) this, right? Anyone know?

But, also to be fair - like you, I maintain my own systems and do not overly depend on the teeming-mass-reality as a derivation of stability. My personal Linux DAW systems, running now for decades, have attained a level of productivity that I would at least hope is represented in the current niveau, vis a vis Popular Linux Distro designed for audio (e.g. pure:dyne, Arch Pro Audio, 64 Studio, UbuntuStudio, et al.) .. for the newcomer, it should of course 'all just work' from boot-up, which I hope is the case. It is for me, anyway: I've expunged pulseaudio from all of my machines, and make do with Jack. My studio uses 48-channels of digital audio, everything-is-a-file .. a working and functional DAW, thousands of plugins, about 12 MIDI devices (synthesizers/effects rack) and so on, and the best thing of all: all source code included. So, yeah .. ;)

EDIT: apropos qjackctl, yeah, apmd:

http://www.tutorialspoint.com/unix_commands/apmd.htm

.. or some such similar thing.

http://pastebin.com/iVAjZzTS

A month ago with Ubuntu 14.04 - paired with a bluetooth speaker, but would not send any audio to it (A2DP) without any indication as to how to diagnose or correct the issue.

It may be standard on some systems, but not, apparently on debian (it's an "optional" package).

And that's part of the point: stuff that needn't be present shouldn't be. systemd's a whole 'nother ball of wax in that regard.

And yes, I'll even allow that Linux audio has been frustrating over the years. But in my case, problems going away had nothing to do with Lennart's work.

I've been with Ubuntu since 2007 and went through the PA transition. I agree it is so much better now. Changing audio sources is easy and faster than on Windows 7. By Ubuntu 12.04 this was stable for me. Like changing from speakers to headsets for a meeting, smooth with PA at least on Ubuntu. Until PA I never thought I'd see audio united on Linux.

On my Debian system youtube videos stop playing sound once in a while (video continues), thought I suppose it's not pulseaudio's fault (so just a general sound problem).

You asked ;-) (I still agree with you that it got better than it once was)

In some sense systemd is more stable in that it's fixing some longstanding bugs with sysvinit, but of course it will have some bugs of it own. If you don't want to deal with that, you could skip a release.

There is a distinct lack of engineering prowess and quality control. It originates at the core GNU + kernel + freedesktop teams and waterfalls down through the distribution houses.

That's the problem and it's endemic within Linux.

Imagine you were an architect of buildings. Your day to day job is to design mundane strip malls and gas stations. You have building code on your side for much of the process. As long as you don't violate the regulation, you at worst can only make an inconvenient building, but not a dangerous one.

But imagine instead that you're building a large office building every six months, your clients demand you don't reuse any design principles on your future clients, and you not only lacked the building code, but also 1/4th of the heavy machine equipment, 1/2ths of the tools, and 3/4ths of the raw materials. I don't just mean you don't have them in stock, I mean nobody has invented them yet. And of the ones that have been invented, we don't even know all of their material properties, say nothing about what material properties we should be looking for. Will this particular bolt we are using with these particular cross beams hold up to the stresses placed on them? The answer is unknown, and I'm some cases unknowable.

It's really easy as a user to say, "they should have tested this more". While strictly true that more testing may have found your issues ahead of time (presuming the right tests were done), it is inefficient engineering to exhaustively test things. Even mechanical engineering bases a lot on statistical modeling, which will always, always have corner cases that don't match reality.

In the real world, people had to learn the hard way about things like lightning rods and sacrificial electrodes. They didn't come about from "testing during development". They came about from testing live, and seeing which buildings and boats did or did not burn down or sink. That's not bad engineering. That's just the nature of unknown problems.

What the general state of affairs shows is the following traits:

1) There is no thought and research going into the design of a piece of software. Ergo, we do not learn from past mistakes.

2) There are isolated individuals writing vast swathes of software which are trusted unconditionally. Ergo, we do not learn the benefit of multiple eyes on a problem, review and discussion.

3) We assume that software is correct from one person's viewpoint and opinion. Ergo, we do not test software properly nor cover those tests with objectives.

4) We work to deadlines, not quality objectives. Ergo, we trade quality for tolerance from others.

In this case someone came along and didn't think about the problem, didn't work with others, assumed they were unconditionally correct and chose tolerance over quality.

To use your analogy, they're now selling stainless steel lightning rods (poor conductivity), are the only vendor of them, are a vocal marketing front and houses are catching fire everywhere.

Or more specifically, in one example, the entire process was above and from the author to the distributor, no one even noticed that loginctl doesn't work properly.

On your points:

#1 is patently false, to the point of being extremely insulting. You've lost all sympathy from me at this point. Go peddle your baseless opinions somewhere the audience doesn't know better.

#2 is also laughably false, as that is the entire freaking point of open source and often considered the greatest strength of Linux. You think because your highly qualified opinion wasn't consulted before you had to spend a whole four hours, OMG on a problem that means that no review or discussion was done?

#3 is false again, because software is tested. You use the word "properly", so I will sit here and wait for you to bestow upon us your great wisdom on what we could be doing better.

#4 is false on both presumptions that software is not built to quality standards instead of deadlines, or that other fields are not dictated by deadlines.

When I say tested properly, I mean tested completely. If you miss an entire functional unit of the software and a client reports it as broken, its pretty obvious what the problem is.

Our senior software guys sat down for the other four hours and presented all our findings together and cumulatively said "we're not supporting that shit; we can't trust it".

Regarding #4, it's plain to see that RHEl was released with a broken systemd implementation due to deadline...

I am still in the learning phase, but even I know that complete testing of any complex software is practically impossible.

So, how do you guarantee completedness in "proper" testing? I know you can't without redefining the word "completely". What's your definition?

Also see Impossibility of Complete Testing[0] by Cem Kanen, co-founder of http://www.associationforsoftwaretesting.org/

[0] http://kaner.com/pdfs/impossible.pdf

When your system consists of functions "A, B, C, D", I'd expect to see test suites for "A, B, C, D". In this case there were test suites for "A, B, C". The client found D therefore the test suite was incomplete.

Now if a bit of A, B, C or D suites were missing that would be different and entirely expected.

As for testing--notice that when a lot of people here are reporting issues with systemd/pulseaudio, their reports are pretty much dismissed out of hand, or they're told "no, you've done something wrong".

For #2, a lot of times somebody with the right political position (say, Lennart at Redhat) or just the ability to shout louder and longer than anyone else will get something put in, regardless of technical advantage. Don't even try to claim otherwise.

1) government contractors are so fun when they get their software required to deal with certain parts of government

Honest question: I've been using sysvinit for a very long time and I have no concept of what those bugs might be.

Assume server with lots of processes.

Service A starts writes PID to disk, lets say 123. lots of processes start and stop as it goes along and does its work. Service A crashes/stops working PID 123 gets reused by a new process SysAdmin comes and hits /etc/init.d/ServiceA restart shell script calls kill 123 which was a totally different process now not at all related to ServiceA.

etc...

Clean unmounting not depending on timeouts to be high enough.

etc...

Not starting a database before the filesystem with the database files is mounted.

etc...

I created a specialized FUSE filesystem to deal with this. Processes create PID files in it, but when they die, the filesystem automatically removes them.

Code: https://github.com/jcnelson/runfs

The Readme is rather sparse, could you add an example how to use it from a init shell script?

Although the Capsicum model (in FreeBSD, slowly getting into Linux) where you can have file descriptors for processes is another different model.

This was solved decades ago with numbered init symlinks:

  K20postgresql
  S20postgresql

So I would say it has been hacked around for decades. Not cleanly solved. But I am not the best informed here so please add more details about how numbered init symlinks guarantee file system being there before a service is started.

It took a long time and a giant ecosystem to get where Linux is today at big enterprises. OSes are commodities in that space. They are not commodities in many other spaces though (e.g. startups, HPC, science, etc).

Whoever is pushing for this is an idiot then. Verisign, for example, has had 100% DNS uptime for the .net, .org, and .com root servers for ~15 years because of their mixed environments. In every one of their POPs they tend to have at least two racks of equipment with:

   * 2 different brands of load balancers
   * 2 different brands of firewalls
   * 2 different brands of switches
   * 2 different brands of servers
    ** servers are from different hardware generations
   * 2 different OSes (Linux and FreeBSD)
   * a choice from 3 different DNS server software

This is how you run a reliable global-scale service. Anyone who plays the "it's just easier if we all use ____" is in for a big surprise when their entire infrastructure is at risk due to one bug.

As long as a sufficient fraction of servers at a sufficient fraction of Verisigns clusters has an uncorrupted set of data and is able to serve responses, Verisigns TLD zones remain up.

Pretty much the only thing that can go wrong in that case, assuming you have safeguarded the integrity of the zone is bugs in components outside their direct control.

It makes 100% sense for them to focus their efforts on ensuring diversity, because the class of problems that can solve for them makes up an unusually high percentage of the possible failure classes, and the nature of the service also means that most of the potential problems that this can cause is only likely to take out some proportion of their capacity that still leaves them with a functional system, so the potential benefit is higher for them than for most with a heterogeneous, and the potential risks are lower for them than for most.

For Google and Facebook, the systems are so much more complex that the tradeoffs are vastly different.

Which excludes the vast majority of functionality of Google/Facebook, and most other major web properties.

There are very few heterogenic systems in the enterprise. That is an objective, but the main thing is that we deliver what we're paid to deliver by choosing an appropriate platform. We have Solaris, zSeries, Linux and Windows. We just got rid of AIX.

As for minor differences, FreeBSD has a lot of much bigger wins than people realise at first glimpse. The differences are far from minor. For example:

ZFS, dtrace, rctl, a scary good IP stack, virtio support, documentation that doesn't suck, a POSIX base, LLVM/clang, a MAC framework that doesn't suck, OpenBSM, CARP and a pile more. Oh plus an automated deployment story that is pretty tidy.

Sure we can replicate some of these on CentOS 7 for example with similar tech but the above are a million times more cohesive.

Unfortunately when the first and second time differ even though identical (recorded) steps were performed, one has to ask the question: why and can I trust it?

My rule of thumb is "search for the problem on Google. If nothing comes up, maybe something is wrong on my end".

Did you find any results or reported bugs similar to what you experienced?

Yes there were other mentions of it with notes to it being fixed in a later systemd drop, which we can't deploy because RH/CentOS don't ship it. I think one of our guys raised a case with RH but I was dragged off onto something else then.

I have my share of reservations about systemd (and PA), but thought that it might be worth pointing out that "known good" hardware A with software X, doesn't have to mean hardware A is all good, just that A has no bugs/errors not exposed when running X. So Y comes along (new kernel, drivers?) with entirely new code - and suddenly things behave erratically.

I don't have the error on my phone which I'm on at the moment but it threw a dbus error with no debug info.

I imagine 10 years ago you would be the person complaining that GCC segfaults randomly during compiling Linux kernel, complaining that it's not "tested completely". While the segfault was caused by CPU overheating (not cooled properly) and flaky memory (causing bits to flip).

Just because a problem is unusual, intermittent, or only affects one person doesn't mean it's not a regular old software bug. And in my experience, it almost always is. And once you do debug it, you often (but not always) understand why it was intermittent, under what conditions it happened, and why you were the only person that saw it.

Nope. Not that.

We don't buy crappy hardware or not test it.