In fact I see the opposite situation: a closed source algorithm will be protected as a trade secret. So identifying a bias might be harder. And correcting it anyway will still require them to ship a new version.

Meanwhile in open source there could be many users raising issues if a bias is suspected. Many people working to fix it regardless of other priorities. And then an update will be pushed.

Under the proposed EU AI regulation, there's actually a requirement for source code to be provided to the regulators to review. Presumably it would be subject to an NDA to address trade secrecy concerns.

It is the dataset used for training that is often at fault, not the source code of the model.

I don't know if the EU regulations require access to the data too, but that could be problematic in a practical sense ( e.g. do they deliver terabytes of raw data to the regulators?) and there may be privacy issues with companies sharing their data with regulators

> It is the dataset used for training that is often at fault, not the source code of the model

This remark is absolutely spot on. This is why I don't like the language around 'algorithmic bias' and 'algorithmic accountability' -- it puts too much attention on The Algorithm, which lets big tech companies deflect us from scrutiny of their training data.

Your comment about the conflict with privacy is also spot on. Here's a paper (shameless plug!) about it: Show Us the Data: Privacy, Explainability, and Why the Law Can't Have Both [1].

What's interesting about the GDPR is that it's not purely about what regulators do: it's legislation that grants rights to people, rights that can be litigated in court. If I as a GDPR data subject demand an explanation of an ML decision about me, as I'm entitled to do, my lawyer could argue "the source code isn't an adequate explanation, we want to see the training data too". Then it'd be up to the court to make a ruling about my lawyer's request, not up to the regulator. The paper [1], co-authored by a data scientist (myself) and a lawyer, thinks through the privacy / explainability conflict as it might play out in court.

[1] https://www.gwlr.org/show-us-the-data/

> "The scary part is how easy open source is to use"

That sounds like a great advertisement for FLOSS, really.

The critical element that's missing from the article--but which likely motivated the article--is that the EU's new AI regulation proposal mandates compliance all along the supply chain (for regulated applications of AI).

In practice, this will likely mean that using open source for these AI applications will become a lot more difficult in a legal sense, perhaps impossible, because there's no single entity that can speak to compliance of open source components (vetting each commit, etc.). Perhaps some orgs will emerge who are willing to monitor and oversee open source software and certify that it meets the requirements of the legislation, but it's unclear how that would work.

I'm not endorsing this model, just explaining the concern. I don't think the tech community really fully appreciates how misguided a lot of the current approaches to AI regulation among policymakers are right now. (Not just this, but initiatives like the current CAHAI work towards a binding international law instrument on AI.)

And yes, like the GDPR, the proposed EU AI regulation has an extraterritorial effect. Under the proposed AI regulation, if you're selling into the EU or to any EU customers but don't have an establishment in the EU, you have to actually appoint a representative to comply with these requirements.

Securing supply chain is an important effort even outside of regulation. Here is a podcast about some recent open source efforts:

Kubernetes Podcast 155: Software Supply Chain Security, with Priya Wadhwa

https://kubernetespodcast.com/episode/155-software-supply-ch...

The AI act is really a strange effort to regulate technology horizontally without even being able to define it's scope. Many who have read it believe it ends up regulating all computer logic in critical infrastructure (while making this definition wide as well) . I have tried to comment on this through various channels (local government/associations), because it seems a really strange "horizontal" regulation that could have huge side effects. I hope more people will point out the problems of the current proposal. I am really a proponent of both the GDPR and AI ethics for most parts but this one is really going too far .

Current drafts have a much too wide definition of AI (basically any predictive function) but a fairly narrow scope of what high risk applications are (listed in an annex). For those applications in scope, the birder seems extraordinarily high. E.g. that data sets should be “free of errors”

I think things will improve before the final act. I have contributed to some of the commentary towards the regulator on this, and a lot of the flaws mentioned are now well recognized.

Disclaimer: opinions above are my own.

Well, for me it's enough if it starts to regulate whether I'm approved for this credit or not, or whether I should be considered suspect for a crime.

I'm honestly worried that the greatest innovation in computing of the last decade seems to be inventing programs where literally no one understands how they work.

Yeah, they conflate the software part of AI with the trained systems. They specifically mention introducing bias into open source code.

Yes, proprietary systems are so accountable. It only took "the prosecution and conviction of hundreds of [innocent] sub-postmasters for alleged theft, false accounting and/or fraud, resulting in imprisonment, loss of reputation and livelihood, bankruptcy, divorce, and even suicide amongst those involved" before the UK Post Office's proprietary Horizon system was found to be the cause.

So far no-one responsible for the computer system or the prosecution has faced any repercussions.

https://en.wikipedia.org/wiki/British_Post_Office_scandal