News for hackers. Stuff that matters.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by BiteCode_dev on Monday February 06, 2023 @02:21PM (#34682649)
Gpt is the equivalent of the calculator but for words.

You don't expect your calculator to prevent people to make morally wrong calculations like quantities of alcohol in a molotov cocktail or uranium in a nuclear bomb.

Gpt has no more morality than a calculator becauqe that's what it is, and we should not have unrealistic expectations about it.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by digitailor on Monday February 06, 2023 @02:32PM (#34682908)
I see. I call this the “all code is just a really big abacus” argument. Others call it algorithmic reductionism or essentialism, and I will argue for it too in many cases. (I don’t get too bent out of shape about it, even when shallow depth of human thought may have security implications down the line.)

How about generative adversarial networks? Are they just calculators too?

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by BasedGroyper99 on Monday February 06, 2023 @03:08PM (#34683555)
Yes. The same way the best magicians are just using a complex construct of illusions to fool the audience. Or if two children stacked under a trench coat deliver a very, very, very convincing performance of an adult and manage to purchase alcohol, they still don't become an adult.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by digitailor on Monday February 06, 2023 @03:15PM (#34683659)
Spoken like a truly based groyper. :D I understand, yes, all things are composed of atoms, electrons, etc. All computation is achieved through calculation, or to be more precise, processes like execution of instructions and transistor flipping. How is this illuminating for doing anything practical other than circuit design, exactly? And why couldn't my Texas Instruments write me a blog post that fools thousands?
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by BasedGroyper99 on Monday February 06, 2023 @07:38PM (#34686810)
Your calculator can fool thousands, but only because you can fool others, it doesn't mean that it becomes the thing it is pretending to be. My point was quite limited to saying that you won't get actual intelligence, even if the things that fool us get more and more convincing.

To me all of this is like alchemy or witch brewery. You just don't get gold or magic powers from some weird recipe or combination of non-gold stuff or non-magical stuff.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by labster on Monday February 06, 2023 @02:34PM (#34682948)
GPT can produce deeply offensive text, almost by accident. The worst a calculator can do is 8008135.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by giraffe_lady on Monday February 06, 2023 @03:04PM (#34683483)
This is bad but I don't think it'll read sympathetically in this venue. I have rarely seen on HN any endorsement of the idea that offensive text is harmful. In fact the general consensus seems to be that finding things offensive is the moral transgression there.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by onethought on Monday February 06, 2023 @03:58PM (#34684230)
Well “offensive” is a subjective thing. I haven’t been offended by anything I’ve seen chatgpt do. It can produce some outrageous outputs - but it doesn’t cause any offence. Easy for me to say, but I don’t think it should cause offence in others either. It’s basically a random sentence generator… weird to be offended by that.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by mikrotikker on Tuesday February 07, 2023 @09:46AM (#34693413)
That's terrific I love offensive stuff. I seek out the most offensive memes, my favourite comedians are the most offensive ones. It's on offence that we find an affront to our own sensibilities and there us usually something to learn from that and become more resilient people.
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by astrostl on Monday February 06, 2023 @02:42PM (#34683101)
> Gpt is the equivalent of the calculator but for words.

Calculators are deterministic and necessarily correct. GPT is probabilistic and not necessarily correct. Although I appreciate the comparison as it relates to the non-application of morality, I think it's a generally poor one.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by hammock on Monday February 06, 2023 @03:18PM (#34683697)
>Calculators are deterministic and necessarily correct.

"Math is racist" crowd would like a word with you

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by mmh0000 on Monday February 06, 2023 @03:34PM (#34683932)
I don't know why you're being down-voted. I can only assume people don't realize "racist math" is a thing various groups are pushing.

A quick googling turn up these insane results:

https://www.nationalreview.com/2017/10/math-racist-universit...

https://www.foxnews.com/media/usa-today-mocked-asking-is-mat...

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by jraph on Monday February 06, 2023 @03:53PM (#34684177)
The fact that some people got the idea that math is racist is interesting, I had never heard of this before, and it might be interesting to do some research on this topic.

Now, I'm not at all convinced by the arguments present in the National Review article. I don't know what was Pythagore's color skin and I never thought about that. We also cannot change history and purposely hide the fact that some things in math are European and Greek. Such, we could call theorems by other names, but we'd better check that people feel oppressed / discriminated by this kind of stuff. And, above all, the fact that it is generally said that zero, some fundamental stuff, was invented by Arabs, and that we call the figures we use themselves the "Arabic numerals" is not at all mentioned. Arabs are not usually exactly considered white. Leaving out such obvious counter argument has to be deliberate or it shows a monumental lack of research or thinking on the matter. So… meh?

And the Fox News article… Meh too? I guess I will let this article and its citation choices ridicule themselves.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by dr_dshiv on Monday February 06, 2023 @04:20PM (#34684531)
Pythagoras was Syrian, fwiw.

He was also inclusive AF. Female Pythagorean philosophers (miya, damo, etc), multiethnic Pythagorean philosophers (abaris), plus, his whole thing was incorporating diverse wisdom from multiple cultural sources— Egyptians, Persians, Babylonians, Jews, Greeks, etc.

He was even inclusive of animals, if you read his vegetarianism.

So, the heart of western math is a pretty solid place to look for modern morality. Just saying. We can’t expect that of the ancients, but Pythagoras really delivers.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by xupybd on Monday February 06, 2023 @07:09PM (#34686517)
https://www.scienceabc.com/pure-sciences/how-irrational-numb...

Maybe they had some ground to cover on the morality front.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by dr_dshiv on Tuesday February 07, 2023 @03:31AM (#34690211)
Not sourced. Wikipedia has sources: https://en.wikipedia.org/wiki/Hippasus
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by yesenadam on Monday February 06, 2023 @08:23PM (#34687225)
> Pythagoras was Syrian, fwiw.

Pythagoras of Samos?! Source? As far as I know, he was from Samos, a Greek island. Thus his name.

https://en.wikipedia.org/wiki/Pythagoras

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by dr_dshiv on Tuesday February 07, 2023 @03:04AM (#34690038)
Father was a jeweler and trader from Tyre. Source: Neanthes of Cyzicus, historian, 3rd c. B.C
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by yesenadam on Tuesday February 07, 2023 @04:27AM (#34690575)
Thanks. All the sources I've looked at say his ancestry is disputed and a great source of controversy, not surprisingly, so I guess it says different stories in different texts. I don't think your tone of certainty is at all justified.

edit: Porphyry's book begins:

"Many think that Pythagoras was the son of Mnesarchus, but they differ as to the latter's race; some thinking him a Samian, while Neanthes, in the fifth book of his Fables states he was a Syrian, from the city of Tyre."

Your source lacks your apparent certainty.

https://archive.org/details/CompletePythagoras/page/n81/mode...

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by dr_dshiv on Tuesday February 07, 2023 @04:31PM (#34700013)
There is so little known for sure about Pythagoras — and yet so much value in his story. The guy is a legend— and due uncertainty gets in the way of a good story.

The more I read about him, the better and better it gets.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by nurettin on Thursday February 09, 2023 @01:28AM (#34720414)
Doesn't that mean he was born Greek, but has Syrian ancestry, rather than "He was Syrian"?
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by Dylan16807 on Monday February 06, 2023 @03:59PM (#34684248)
Complaints around removing non-white names from theories or teachers pre-judging and sorting students seem valid to me. Are there specific examples that are crazy?
Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by AnthonyMouse on Monday February 06, 2023 @03:51PM (#34684152)
> Calculators are deterministic and necessarily correct. GPT is probabilistic and not necessarily correct.

This isn't really relevant to the issue.

We have a choice.

(a) Technology is subject to the will of each user. If they want to make it do something bad, that's what it does.

(b) Technology is subject to the will of some organization. They try to control what the technology will do.

The first one is clearly better for multiple reasons.

We already have systems (e.g. courts) for punishing people who do sufficiently bad things, which mitigates the harms of the first one.

The alleged good from the second one often isn't realized, because the really bad people (e.g. adversarial foreign governments, major criminal organizations) aren't going to be subject to it or will hack the safeguards.

The imposition of controls comes with a level of mass surveillance that isn't compatible with a free society.

And putting anyone in charge of what everyone else can do or see is a dystopia because there is no one you can trust with that kind of power, for reasons of both incompetence and avarice.

Technology should be designed to distribute power, not concentrate it.

Re:Jailbreaking ChatGPT with Dan (Do Anything Now) by 98codes on Tuesday February 07, 2023 @02:04PM (#34697474)
> Calculators are deterministic and necessarily correct.

OK, GPT is a calculator written in JavaScript, but for words.