News for hackers. Stuff that matters.
Re:A Linux Evening by l1k on Friday December 16, 2022 @07:30AM (#34013844)
Thunderbolt devices appear in the OS as a PCIe switch, so you need two additional bus numbers (one for the Switch Upstream Port and one for the Switch Downstream Port). If the device is hotplugged to a port which has run out of bus numbers, you'll get this error message.

Mika Westerberg is constantly fine-tuning the allocation of PCI resources in the Linux kernel to avoid such scenarios. Some recent patches:

https://lore.kernel.org/linux-pci/20220905080232.36087-1-mik...

https://lore.kernel.org/linux-pci/20221130112221.66612-1-mik...

On macOS, it's possible to pause the PCI bus, reallocate resources and unpause the bus:

https://developer.apple.com/library/archive/documentation/Ha... (search for "Supporting PCIe Pause")

We don't have that on Linux unfortunately, so we depend on getting the initial resource allocation right.

Sergei Miroshnichenko has worked on such a reallocation feature for Linux but it hasn't been accepted into mainline yet and he hasn't posted a new version of his patches for almost two years, so the effort seems stalled:

https://lore.kernel.org/linux-pci/20201218174011.340514-1-s....

Re:A Linux Evening by nneonneo on Friday December 16, 2022 @08:28AM (#34014282)
It sounds like the pause/unpause might be the way to fix this properly, since trying to be heuristically smarter sounds like a recipe for never-ending corner case bugs like the OP’s issue.

The patch for pausing and unpausing seems quite reasonable, except that it does require driver support (unsurprising - you’re literally reallocating the resources used by the driver!). I suppose if you had at least a few movable devices then you should be ok in the event of a hotplug event, so you’d have to hope that enough drivers bother to support the feature.

I wonder what is necessary to get people to care about the patch enough to fix it up and mainline it? I suppose the problem it fixes is still niche enough that not so many people are clamoring for the fix.

Re:A Linux Evening by AnIdiotOnTheNet on Friday December 16, 2022 @08:53AM (#34014459)
What is the point in having all of the drivers be open sourced and mainlined if we're not willing to fix them to support this?
Re:A Linux Evening by blueflow on Friday December 16, 2022 @08:56AM (#34014475)
You are free to submit a patch or request a refund.
Re:A Linux Evening by jodrellblank on Friday December 16, 2022 @11:24AM (#34016425)
You are free to comment on the question raised, instead of a dismissive empty ad-hom.
Re:A Linux Evening by arghwhat on Friday December 16, 2022 @12:28PM (#34017400)
While tongue in cheek, the answer is accurate: open source does not mean "free service contract", it means that you can take the code and modify yourself (and preferably upstream the fix).

Patches come both from vendors and users experiencing an issue. Vendors take care of most things, but for esoteric problems you might only have a handful of people experiencing it. The vendor is unlikely to care, so if you do not write the patch or pay someone to do it, who will?

Still better than the competition, where such problems will never be fixed unless it generates sufficient bad PR...

Re:A Linux Evening by jodrellblank on Friday December 16, 2022 @12:44PM (#34017649)
You're doing the same thing, dragging the original conversation off into berating the commentor for not fixing it themselves because you owe them nothing and they shouldn't be so entitled.

> "Still better than the competition, where such problems will never be fixed unless it generates sufficient bad PR..."

The competition comes under "pay someone else to do it".

Re:A Linux Evening by elsjaako on Friday December 16, 2022 @01:31PM (#34018228)
What OS are you using where you can get the vendor to implement kernel features to fix obscure driver issues?

I'm sure there's an amount of money you can throw at Microsoft to get something done. I don't know how much it is, but I'm guessing it's more than it would cost to find a vendor to do it for Linux.

The serious answer to " What is the point in having all of the drivers be open sourced and mainlined if we're not willing to fix them to support this?" is "There are many points to this, but one of them is that it's possible to fix them to support it, if someone wants to put in the effort. It's worth something that it's theoretically possible if you really need it, even if no one else has done it yet.".

The answer "You can do it yourself" is meant to help them understand "Anyone can do it, someone needs to step up to the plate. But it's also true that it costs resources. If you're wondering why no one else has done it yet, it's the same reason you haven't done it yet".

Re:A Linux Evening by fragmede on Friday December 16, 2022 @02:10PM (#34018689)
With enough money you can get that kind of support from any Linux vendor, eg RedHat or Oracle.
Re:A Linux Evening by arghwhat on Friday December 16, 2022 @03:01PM (#34019484)
> You're doing the same thing, dragging the original conversation off into berating the commentor for not fixing it themselves because you owe them nothing and they shouldn't be so entitled.

Ah, this argument again. Yeah, the maintainers owe you nothing, as they have already worked their asses off to give you something for free. You have the right to make polite bug reports and discuss fixes, but no one is entitled to force volunteers to do work.

But, that is not the same thing as everyone having to fix their own shit. 100 million users does not need 100 million developers.

What matters is that the users that have issues can fix issues, and if the issue affects enough people, it will eventually affect someone able and willing to fix it. That is why open source works, but it requires that some people put in the effort, and many learn to do it exactly when they get annoyed by a bug.

So yeah, if you are not willing to wait for someone else to come around and volunteer to fix it, patch it yourself or pay someone else to do it. That's how the system works, regardless of how demeaning you feel this is to non-developers or developers that feel that their time is more valuable than that of others.

> The competition comes under "pay someone else to do it".

Sure, if you have enough money to convince Apple or Microsoft specifically to prioritize fixing your issue (which may be in an unsupported or deprecated configuration) above what else they were doing, which would cost a whole lot more than just engineer and manager time. You have no alternative, as only your specific vendor can make the fix. Realistically speaking, if you had that kind of money you probably already have employed engineers that you could get to fix your open source issues for you and would not be arguing on hacker news about the need to write patches.

For open source, you don't have to convince anyone in particular. Can't convince the first person you try with money? Just ask the next person, anyone can submit the patch.

Re:A Linux Evening by jodrellblank on Friday December 16, 2022 @04:27PM (#34020672)
I think you are mixing two arguments. One is a good, valid, argument which is about how volunteer maintainers don't owe anyone anything, and absolutely don't deserve to be harassed, insulted, coerced, guilt tripped, etc. And the other is is about internet Linux commenters (away from bug trackers and issue lists) replying in ways that close down and end conversation of anything which isn't toeing the 'party line' of how great Linux/FOSS/libre/gratis software/etc. is.

The parent comment by AnIdiotOnTheNet was not in the context of bug reports filed to maintainers, or insulting anyone, or demanding anything specific. The parent of that said that the patch looked good "but" would need driver support, perhaps suggesting that's a showstopper. AnIdiotOnTheNet asked what the point of having open source drivers is if they can't be fixed, or charitably steelmanned read as "the drivers are open so they can be fixed to work with the patch". Blueflow's reply "you are free to submit a patch" is technically correct, but low value - few people on HN aren't aware of that. The following "or request a refund" is conversation ending, "fix it or shut up, stop talking about it".

It's a common reply format on internet Linux discussions which is closer to 'silence wrongthink' or 'cancel culture' than tech discussion.

> "So yeah, if you are not willing to wait for someone else to come around and volunteer to fix it, patch it yourself or pay someone else to do it."

Or ... talk about it, rant about it, 'raise awareness', exercise freedom of speech. "Patch it or shut up" aren't the only options. And look, dkozel replied with a long and technically detailed comment[1] and didn't need anyone jumping in to silence unapproved questions.

[1] https://news.ycombinator.com/item?id=34016094

Re:A Linux Evening by blueflow on Friday December 16, 2022 @05:26PM (#34021432)
Your ideas and intentions might be good and noble, but in the end of the day its the contributors and maintainers who burn out. And from my impression, most people in OSS are already fed up with supporting users. And I'm too. Telling users off like "fix it or shut up, stop talking about it" is the necessary step to protect yourself.
Re:A Linux Evening by runjake on Friday December 16, 2022 @02:23PM (#34018894)
What it sounds like you’re saying is “If you don’t know how to code, GTFO” because in all likelihood the parent made this comment because they’re not capable.
Re:A Linux Evening by arghwhat on Friday December 16, 2022 @03:05PM (#34019553)
Not quite. If you are a non-developer not patient enough to wait for others to volunteer, or are a developer thinking that your time is somehow more valuable than those of the maintainers, then GTFO. :)

Waiting patiently and politely reporting bugs is a fine strategy: If a problem affects enough people, it will eventually affect a developer capable and willing to fix it. If you want it go faster, you will have to get your hands dirty - many contributors acquired the skills exactly because they were annoyed by an issue and decided to fix it.

Re:A Linux Evening by jodrellblank on Friday December 16, 2022 @09:15PM (#34023612)
Where did AnIdiotOnTheNet's comment include not being a developer, not being patient, or thinking their time is more valuable than others?

Why is it so much more important to put someone in their place and win internet points with ad-homs in the "community" which supposedly values freedom?

Re:A Linux Evening by blueflow on Saturday December 17, 2022 @06:04AM (#34026775)
"Whats the point in ${enormous amount of work others did for me for free} ..." is never a polite move. Neither is your second sentence.

You two are just rude. No amount of freedom forces other people to bear with that. And if you think its not rude, still, the world is large enough to avoid each other.

Re:A Linux Evening by jodrellblank on Saturday December 17, 2022 @07:11AM (#34027113)
> ""Whats the point in ${enormous amount of work others did for me for free} ...""

That is the least charitable interpretation of it, and I think not at all what it actually says.A much better reading of it is, paraphrased:

"The PCI-e patch is nice, but it would be a dealbreaker waiting on driver support."

"The drivers are open source and checked into the same tree, so there's no dealbreaker there in waiting for vendors or coordinating with third party organisations and their release schedules {and that's the point of desiring open source drivers, so the system isn't hobbled by binary blob drivers and vendor release schedules}".

Re:A Linux Evening by blueflow on Saturday December 17, 2022 @07:32AM (#34027215)
> dealbreaker

Very poor choice of words. If the deal is bad, request a refund!

I understand what you are trying to say, but i think that's just entitlement.

Just accept that, as a non-contributing OSS user, you have zero leverage over which features other people pour their energy into. If you do, that's a charitable exception and happens at the generosity of the one doing the work for you.

Edit: Let me quote the licensing terms that you agreed to and gave you permission to use the linux kernel at all:

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.
Re:A Linux Evening by jodrellblank on Saturday December 17, 2022 @05:12PM (#34032816)
Can you get off your high horse about finding someone to punish for some imaginary sin which hasn't even been committed in this thread, and respond to some of the things I said and the points I made?

The "deal" in question was imaginary person A offering patches to the kernel to fix the articles's PCIe allocation, and the Kernel maintainers hypothetically refusing the patches on the grounds that it would break compatibility with drivers and so it cannot happen. Then the comment that the drivers could be updated to match, so the patch could hypothetically happen. There is NO entitlement anywhere in this imaginary scenario, there is no demand for anyone to write such patch, no expectation that someone get right on updating the drivers, no insinuation that someone owes a review of such a patch, nothing that you are so het up about has happened or been implied to happen, been demanded, expected, requested, suggested or implied.

> "Just accept that, as a non-contributing OSS user, you have zero leverage over which features other people pour their energy into."

Where did you come up with the idea that I am a non-contributing OSS user? Because it drives your superiority fantasy, I suspect, where "putdowns of the inferior" are the order of the day.

Re:A Linux Evening by mijoharas on Saturday December 17, 2022 @06:30AM (#34026910)
I wouldn't exactly call the comment you're replying to an ad hominem attack.
Re:A Linux Evening by jodrellblank on Saturday December 17, 2022 @06:39AM (#34026957)
Why not? You don’t think it changes the topic from being about the content of the comment (open source drivers) to being about the person who wrote the comment (what they should and shouldn’t be permitted to say based on how much they paid or didn’t pay)?
Re:A Linux Evening by blueflow on Saturday December 17, 2022 @07:51AM (#34027327)
If the problem lies in the entitlement of a person, an appropriate response is going to be an ad hominem, and rightfully so.
Re:A Linux Evening by jodrellblank on Saturday December 17, 2022 @05:16PM (#34032854)
You have not shown any signs of this spectre of entitlement haunting your every comment. If you had, an appropriate response would be educational, helpful, or perhaps a link to Rich Hickey's Gist[1].

By your comments, it would be appropriate to ad-hom you now for your apparent entitlement to controling other people's speech about OSS, right?

[1] https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...

Re:A Linux Evening by blueflow on Saturday December 17, 2022 @07:27PM (#34033779)
Its simpler: your attitude is bad, i told you off, you have no intentions to change your attitude, neither do i.

And if thats "controling other people's speech" to you, then yes, your attitude is the problem. Realistically, i'm just some random dude on this forum, telling you off is the worst thing i can do to you.

Enjoy your refund.

Re:A Linux Evening by jodrellblank on Sunday December 18, 2022 @08:33AM (#34037486)
> "you have no intentions to change your attitude, neither do i."

Multiple times now I have pointed out that I have not shown the behaviours you are accusing me of. The difference of who has intentions to change is that you are wrong.

> Enjoy your refund.

You are free to link to the place in this thread where I personally said I was unsatisfied with some piece of software, or unsatisfied with the work someone was doing on it, or in any way mentioned desiring a refund or expecting more work? I think you won't be able to find such a place, so your 'clever comment' falls flat.

Re:A Linux Evening by bee_rider on Friday December 16, 2022 @03:39PM (#34020026)
Possibly it is hard, tedious, or the people able to fix it don’t think it is worth the effort.

Open source projects rely on volunteers mostly so it isn’t like there’s some outside force to appeal to. If nobody volunteers a solution, then it isn’t important enough to solve. The point is that, if it were important enough to fix, anybody with the requisite skills could do so.

Re:A Linux Evening by lostdog on Friday December 16, 2022 @08:50PM (#34023410)
> What is the point in having all of the drivers be open sourced and mainlined if we're not willing to fix them to support this?

With open source and mainlined drivers, it's very difficult to change all the drivers and ensure they work.

Without open source and mainlined drivers, it becomes impossible.

Re:A Linux Evening by planede on Friday December 16, 2022 @09:31AM (#34014858)
Maybe it can be introduced gradually, making the reallocation an optional feature that a driver might support. Then drivers can independently implement the resource reallocation feature.

Mainline drivers can move gradually. If they want to be nice for out-of-tree drivers then they can describe a timeline for deprecating and removing the support for non-reallocating drivers.

Re:A Linux Evening by bombcar on Friday December 16, 2022 @11:13AM (#34016246)
Would a workaround be that whenever the kernel detects this happening (and it did, it dmesg printed it) that it somehow increases an internal counter so on next reboot there will be more resources?

This would require the kernel being able to either update its own command line somehow, or having some permanent storage somewhere it could store it.

Or this could all be done by systemd - detect that message, increase the resource, next reboot will fix it.

Re:A Linux Evening by smashed on Friday December 16, 2022 @07:15PM (#34022537)
Kernel state does not survive reboots afaik.

That would need help from userland, which is not involved in the early boot process.

You could I guess change kernel init parameters and save that in your boot loader, but that is very hackish.

Re:A Linux Evening by l1k on Friday December 16, 2022 @01:34PM (#34018268)
The PCI resource allocation code is fairly intricate and everyone is scared that changing it may cause regressions. Sergei's patch set is quite intrusive and it would be necessary to somehow break it up into smaller pieces that are slowly fed into mainline over several release cycles, always watching out for regression reports. So, the problem is known, but the engineers working on PCI code in the kernel are given higher priority stuff to work on by their employers, hence the issue hasn't gotten the attention it deserves.

Actually I forgot to mention there's another solution: A PCIe feature called Flattening Portal Bridge (PCIe Base Spec r6.0 section 6.26). That was introduced with PCIe 5.0. It's more likely that FPB support is added in mainline than the pause/unpause feature. It's supported by recent Thunderbolt chips and it's an official feature of the PCIe standard, so companies will prefer dedicating resources to it rather than some non-standard approach.

Re:A Linux Evening by sylware on Friday December 16, 2022 @06:04PM (#34021823)
In the dynamic use cases, the PCIe specs is kind of shabby on the addressing space: it is theorically fixed by FPB.

I guess this is sorry for those niche hardware use cases.

Isn't FPB into PCIe 4.0? (I am not a SIG member, cannot read the specs).

Re:A Linux Evening by sylware on Saturday December 17, 2022 @11:47AM (#34029465)
I meant, I know about PCIe addressing (from the web, linux code, and a book I read years ago), but I cannot read the modern specific FPB specs.
Re:A Linux Evening by mrb on Friday December 16, 2022 @05:10PM (#34021197)
The key takeaway, especially for the author who wants things to "just work", is that he should have used a USB external disk, instead of a Thunderbolt external disk. OSes tend to have more issues with Thunderbolt disks (such as this issue you explained in details) than with USB disks, because the latter is more common, so more corner case bugs have been eliminated.
Re:A Linux Evening by bhewes on Friday December 16, 2022 @09:24PM (#34023673)
Yeah, first thing I thought was oh that drive looks esoteric.
Re:A Linux Evening by fabiensanglard on Saturday December 17, 2022 @03:36AM (#34025990)
Any book or documentation you recommend to read to someone interested in getting involved?