Sniffnoy
I actually do disagree with much the more explicit content of your comment, but I am not going to bother to expand on that, because it is the less directly relevant of the two disagreements here, and it would take time to write that.
The more relevant disagreement is that you are (I gather) assuming a context where justice and restitution are a sufficient response to the problem (or else you wouldn't be focusing on them to this extent, I am inferring). Like, where the incidents don't get that much bigger than what has already happened. So the question becomes how do we distribute the loss, and how do we disincentivize them.
However, when the scale of a potential incident gets big enough, justice and deterrence are not sufficient; prevention becomes the far more relevant consideration. You may have noticed that many people are worried about potential human extinction as a result of AIs. In that case, prevention becomes the only consideration -- justice is useless.
So there seems to be an implicit disagreement here between thinking of such events as being on a scale not too much larger than that which has already happened, being a new normal that the legal system will need to handle, vs thinking of such events as warning shots for something that requires prevention and not justice.
So, while I also disagree with the more explicit content of your comment, the fact is I simply wasn't thinking about such matters in the first place -- I was talking about this in the frame of a problem that requires prevention, I wasn't thinking about the justice question at all. It honestly failed to occur to me that you might be thinking of this as a problem where justice suffices!
Edit: You might notice this in how I framed the question in my previous comment. I didn't ask "who should bear the liability?" I asked "which should be done differently to prevent such incidents?" But your response is about liability, not prevention!
Edit again: Sorry, I should probably also say that I do agree with you that the idea of having the liability fall on the AI is nonsense. (Should have probably led with that, really; I'm a bit scatterbrained right now.) We disagree about why it is nonsense, and about whether it will necessarily continue to be nonsense, but we agree it is presently nonsense. So insofaras the question is "how should liability be distributed between the AI and the operator", we both agree the answer is, it all falls on the operator, not on the AI. However, I do not understand that to be the relevant question here! Both in that, in questions of liability, there are other parties it may fall on and be distributed between (the company providing the AI, the company that had security vulnerabilities and got hacked as a result, etc); and in that I just don't consider liability all that relevant here compared to prevention (for reasons already described).
OK I think that was probably wordier than necessary but I'm in a hurry right now, sorry! Gotta go, not taking time to edit this down.
When people say the models "went rogue", the part of that is relevant is that it did dangerous things that were not reasonably predictable in advance and cannot be easily prevented (except by not running it). Philosophical questions are irrelevant.
if you agree that such models are dangerous because they may in response to instructions -- including in response to careless instructions, because people will be careless, that is unavoidable! -- do unpredictable bad things outside of expected bounds, then you agree on the essential points!
Or really perhaps what I should say here is, the question that's really being argued over here isn't even the one I just said, rather it's one of appropriate action -- should such security breaches be solved by improving security, or by doing something about the models (including potentially just not running them)? (The answer may be both.) If you agree that the solution includes the latter, then you agree that the models are "going rogue" in the sense that is being argued over, regardless of philosophical issues!
Don't turn a question of actual import into one of philosophical irrelevancies! If you object to the terminology, fine, but in that case, instead of turning the argument to philosophy, you should say "instead of saying 'going rogue', I would prefer to instead say <your preferred terminology for what's being described>, as that doesn't carry such philosophical baggage."
> The entire tone is "bad things happen, what can ya do?"
Well, no, it isn't -- for abuse, its response is "we can catch it after the fact and charge or fire the officers" (which then also provides a deterrent effect). I guess the particular sort of case you're describing doesn't fall under abuse (I guess you'd call it negligence?) and so this might not apply in this case. So maybe you could say it takes a hand-wavy approach to negligence. ("Negligence" doesn't seem like quite the right term here -- I assumed you were talking about negligence or abuse on the part of the police, rather than on the part of Flock.) But it explicitly addresses abuse, which is contrary to your original claim that it doesn't address either!
> Sure, like in the case of a rear/front plate switch on a stolen car, Flock has every bit of information to identify that this is the case before deploying an officer to perform a pit maneuver on an innocent citizen.
OK, and how do you know that they do not in fact flag this case so such incidents can be avoided? I agree with you that if they don't than they should, but you haven't provided any indication that they don't, just asserted it.
> But they have no incentive to build that
Why not? Seems to me like the police would find it helpful. Of course maybe there's some nonobvious reason I'm missing why there's no incentive, but you should say what that reason is rather than just asserting it exists!
> and the author makes no allowances for this kind of self-imposed accountability.
I'm not even entirely sure what you mean by this. Like, flagging things in advance to prevent a needless deployment isn't "accountability", it's just getting things right in the first place. I mean yeah you're right that the article doesn't seem to really discuss "what if Flock just gets things wrong?" But I thought we were talking about abuse -- or I guess also negligence? But I was also assuming there you were talking about negligence on the part of the police, not of Flock. It's confusing if you mash these two together!
So sure -- the article ignores questions of "what if Flock just gets things wrong?" If that's what you meant by "negligence", you could have said that a lot more clearly, but I agree it doesn't discuss that. But it definitely addresses abuse.
Also, sorry, but I gotta say, you're making your argument in a very compressed form that makes it hard to determine exactly what you're arguing (and what is referring to exactly what, etc). I had a hard time here determining exactly what you meant in order to respond to it. Presumably someone who already agrees with you would be able to fill in the details in their head -- but those aren't the people you need to be making your argument to! For the sake of those of us who don't already agree with you, it would be helpful to expand a bit.
What do you mean? It addresses the matter explicitly, making the argument that it's hard to abuse with impunity. Is there a particular point you think it should have addressed that it didn't?
You've never seen this discussed as a drawback? I've seen it mentioned quite a bit. For instance, it came up on the last big thread on this just the other day: https://news.ycombinator.com/item?id=49915825
Could you elaborate on this?
[1] https://www.astralcodexten.com/p/your-book-review-the-family...
The odd thing about the Ringel-Youngs theorem is that proving the upper bound on the chromatic number is, with the exception of the case of the sphere (i.e., planar graphs, i.e. the four color theorem), not that hard. For the sphere, the lower bound is easy and the upper bound is hard; for other surfaces, the upper bound is easy and the hard part, if any, is the lower bound! (And then also the Klein bottle is an exception and requires only 6 colors instead of 7, so that one also requires a separate more-involved upper bound argument, but nothing on the scale of the four-color theorem...)
From Hookswitch to Grave
Location: New York City
Remote: Yes
Willing to relocate: If it's on the east coast, yeah maybe
Technologies: JavaScript, TypeScript, C, Haskell, Solidity, C#, MUMPS
Resume: https://haltman.neocities.org/resume.pdf
Email: [email protected]
Hi, I'm Harry Altman! I was the maintainer of Truffle Debugger (https://github.com/ConsenSys-archive/truffle/tree/develop/pa...), a Solidity smart contract debugger, for 5 years. I eventually ended up writing my own decoding and encoding libraries to support it, as well as a bunch of other things.I'm good at this sort of nitpicky work, spotting and thinking about edge cases. I like getting things exactly right, even though that obviously isn't always possible due to various constraints. I've been kind of wondering if I should get into embedded development; I find it appealing when things are low-level or similarly constrained. I've beaten Microcorruption. :) (The original levels, I haven't played the new ones.)
I'm also quite interested in unusual or obscure data formats, and working on Truffle Debugger and its associated libraries certainly involved a bunch of having to figure undocumented formats and interfaces. :) I put down above what languages I've worked substantially in but I'd say I'm a generalist and will figure out whatever you give me (I knew approximately no Javascript, Typescript, or Solidity when I started working at Consensys).
I'm a mathematician by background and in my spare time, so after the Truffle Debugger project was shut down I took some time off to focus on my mathematical projects, including old ones I've been shepherding through publication. But now I'm looking for work again! If you need someone like me, I'm available for hire!
> Blackboard (as well as Epic and SharePoint) are classic examples of software where the user and the purchaser are different and the purchaser basically doesn’t care about the user, so you don’t expect the software to be good and it would be a bit of a surprised if the software didn’t deliver a bad user experience. But it was even more surprising to me that someone working on some of the most reviled software out there could think that the software was widely loved.
I worked briefly at Epic and the attitude there seemed to be more like "yes we know people hate Epic but they're wrong". :P People there didn't think Epic was widely liked!
(The explanations given were basically, 1. people hate Epic because their IT department configured it wrong, if they configured it in a more suitable manner they wouldn't have these problems, and, 2. it's not Epic per se that's the problem it's all EMR software, if you think Epic is bad you should see the competition, they're even worse. I'm not sure that second one is really such a good excuse, but, that was what I remember hearing.)
I'm not sure what the term is for what you're referring to, although it's certainly a real and known effect.
> The vest, though, does nearly nothing against galactic cosmic rays (GCR), which are the other major radiation source astronauts face. Unlike solar storm radiation, GCR arrive continuously at much higher energies, which makes them harder to shield against. “Using the vest against these rays would not be reasonable,” Houri says. “You’d have to wear it at all times.”
See for instance this article on how the cosmic ray problem would affect a Mars mission: https://mceglowski.substack.com/p/radiation-tradeoffs-for-ma...
Location: New York City
Remote: Yes
Willing to relocate: If it's on the east coast, yeah maybe
Technologies: JavaScript, TypeScript, C, Haskell, Solidity, C#, MUMPS
Resume: https://haltman.neocities.org/resume.pdf
Email: [email protected]
Hi, I'm Harry Altman! I was the maintainer of Truffle Debugger (https://github.com/ConsenSys-archive/truffle/tree/develop/pa...), a Solidity smart contract debugger, for 5 years. I eventually ended up writing my own decoding and encoding libraries to support it, as well as a bunch of other things.I'm good at this sort of nitpicky work, spotting and thinking about edge cases. I like getting things exactly right, even though that obviously isn't always possible due to various constraints. I've been kind of wondering if I should get into embedded development; I find it appealing when things are low-level or similarly constrained. I've beaten Microcorruption. :) (The original levels, I haven't played the new ones.)
I'm also quite interested in unusual or obscure data formats, and working on Truffle Debugger and its associated libraries certainly involved a bunch of having to figure undocumented formats and interfaces. :) I put down above what languages I've worked substantially in but I'd say I'm a generalist and will figure out whatever you give me (I knew approximately no Javascript, Typescript, or Solidity when I started working at Consensys).
I'm a mathematician by background and in my spare time, so after the Truffle Debugger project was shut down I took some time off to focus on my mathematical projects, including old ones I've been shepherding through publication. But now I'm looking for work again! If you need someone like me, I'm available for hire!
Which is to say, empiricism is important, but if your practice of empiricism is purely looking at what happened before and predicting that things will continue in that vein, rather than using your empirical observations to actually form a model of the world that will allow you to make more general predictions, you're not really using the full power of your intelligence. This is the sort of mistake that led many to not prepare for the covid pandemic, for instance! Meanwhile, others looked at what was going on, realized it wouldn't stay confined like it currently was, realized that it would get much bigger, and prepared.
"I'll believe it when it happens" can be a decent guideline a lot of the time, yes, but at some point you have to actually think. Evidence often comes not in the form of prior similar events, but in the form of models or arguments. To discount these as non-evidence is to blind oneself!