News for hackers. Stuff that matters.
Re:All of ChatGPT's System Prompts by rabbits_2002 on Saturday October 14, 2023 @07:38AM (#37879730)
It is crazy to me that we have actually reached a point where you just tell a computer to do something, and it can
Re:All of ChatGPT's System Prompts by jbotdev on Saturday October 14, 2023 @08:25AM (#37879936)
Technically that’s always been the case. It’s just that now you can tell the computer what to do in a “natural language” like English instead of a “programming language”.
Re:All of ChatGPT's System Prompts by z3t4 on Saturday October 14, 2023 @08:46AM (#37880058)
We have reached a point where the computer no longer does exactly that we tell it to do :P I always though of computers being stupid, but very fast. With AI they are slow, and make human errors :P
Re:All of ChatGPT's System Prompts by loloquwowndueo on Saturday October 14, 2023 @09:11AM (#37880209)
With the right GPU they can make mistakes almost at the speed of thought.
Re:All of ChatGPT's System Prompts by PaulDavisThe1st on Saturday October 14, 2023 @09:16AM (#37880234)
I've been writing software for 35+ years, and the computer has frequently done something other than what I thought I told it to.

Turns out that I'm not as good at telling computers what to do as I'd like to think.

Re:All of ChatGPT's System Prompts by spiderice on Saturday October 14, 2023 @09:32AM (#37880353)
And now we intentionally tell the computer to ignore what it is told if it’s isn’t deemed proper by the mega corps.
Re:All of ChatGPT's System Prompts by taneq on Saturday October 14, 2023 @08:35AM (#37879989)
And if it won’t, you might be able to weasel your way into it with a sob story about your grandma.
Re:All of ChatGPT's System Prompts by dwroberts on Saturday October 14, 2023 @08:36AM (#37879997)
I think this is the point where the field has just entered pseudoscientific nonsense.

If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Re:All of ChatGPT's System Prompts by andrewpolidori on Saturday October 14, 2023 @08:42AM (#37880036)
Interesting! What do you think it's a red flag for? If it's easier to use the prompt why not do that at the level of abstraction that makes sense?
Re:All of ChatGPT's System Prompts by layer8 on Saturday October 14, 2023 @08:55AM (#37880119)
It means we don’t understand the mapping between the model and the prompt-level semantics. It also means we can’t cleanly separate (1) controlling the conversational behavior of the model from (2) having a conversation with the model. It means we have a black box that we can experiment on, but we can’t plan or predict how to control and customize it, nor can we reliably predict and control its behavior, because we can’t accurately reason about it.
Re:All of ChatGPT's System Prompts by kenjackson on Saturday October 14, 2023 @12:01PM (#37881556)
None of your conclusions are true. We can plan and predict just fine with it. It’s not fully deterministic, but still more predictable than people. And there are a lot of things we don’t understand but are fairly predictable, like the use of pain medications. They aren’t 100% predictable, but good enough to be able to administer as part of a well thought out regimen.

And given the fact that we can test it in a somewhat closed system gives us much more ability to predict its behavior than many things “in real life”.

Re:All of ChatGPT's System Prompts by Tactician_mark on Saturday October 14, 2023 @08:44AM (#37880047)
Sure, it's a sign that we don't "understand this stuff properly", but you can say the same about human brains. Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?
Re:All of ChatGPT's System Prompts by fakedang on Saturday October 14, 2023 @09:29AM (#37880337)
> Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?

From an almighty creator's, yes. Direct telepathic communication is much more efficient compared to spoken language. Just look at the Protoss and where they went, leaving us behind :-(

Re:All of ChatGPT's System Prompts by LadyCailin on Saturday October 14, 2023 @08:58AM (#37880132)
We have no choice but to have and use our brains, not so with LLMs. We don’t have to start building core technologies off of fundamentally flawed models.
Re:All of ChatGPT's System Prompts by diydsp on Saturday October 14, 2023 @09:14AM (#37880224)
Great point. Btw: The problem is corporate irresponsibility:

When self-driving cars were first coming out a professor of mine said "They only have to be as a good as humans." It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice. Corporations dissipate responsibility by design. When self-driving cars kill, no one goes to jail. Corporate fines are notoriously ineffective, just a cost of doing business.

And even without the legal power, most people do try to drive well enough to bit injure each other which is a different calculus from prematurely taking products to market for financial gain.

Re:All of ChatGPT's System Prompts by cdogl on Saturday October 14, 2023 @10:09AM (#37880587)
Very succinctly put - this captures my view but I couldn’t have put it in these words; thanks!
Re:All of ChatGPT's System Prompts by kenjackson on Saturday October 14, 2023 @11:55AM (#37881483)
Human errors are not corrected by discipline and justice. Drunk driving is still a huge problem.
Re:All of ChatGPT's System Prompts by WoodenChair on Saturday October 14, 2023 @01:53PM (#37882626)
The top 3 causes of death by vehicle accident in USA are [0]:

- DUI

- speeding

- distraction

In other words all human errors. Machines don’t drink, shouldn’t speed if programmed correctly, and are never distracted fiddling with their radio controls or looking down at their phones. So if they are at least as good as a human driver in general (obeying traffic laws, not hitting obstructions, etc.), they will be safer than a human driver in these areas that really matter.

What do you care more about—that there is somebody specific to blame for an accident or that there are less human deaths?

0: https://www.idrivesafely.com/defensive-driving/trending/most... and many other sources you can find

Re:All of ChatGPT's System Prompts by diydsp on Saturday October 14, 2023 @04:11PM (#37883844)
Under corporate control safety spirals down to increase profit. See: opiods, climate change, pesticides, antibiotic resistance, deforestation, and privacy. 50 years from now self-driving cars will be cheaper and more dangerous. Human driving misbehavior will still be disincentivized through the justice system, but corporations will continue to avoid individual responsibility for dangerous programming.
Re:All of ChatGPT's System Prompts by bliteben on Sunday October 15, 2023 @08:26AM (#37889185)
We could likely put 80% of cars on rails just as easily
Re:All of ChatGPT's System Prompts by robswc on Saturday October 14, 2023 @02:26PM (#37882909)
They only have to be as good as humans because that's what society deems an acceptable risk.

I do think the point about how companies are treated vs humans is a good one. Tbh though, I'm not sure it matters much in the instance of driver-less cars. There isn't mass outrage when driver less cars kill people because that (to us) is an acceptable risk. I feel whatever fines/punishments employed against companies would only marginally reduce deaths, if that. I honestly think laws against drunk driving only marginally reduce drunk driving.

I'm not saying we shouldn't punish drunk driving... just that anything short of an instant death penalty for driving drunk probably wouldn't dissuade many people.

Re:All of ChatGPT's System Prompts by Szpadel on Saturday October 14, 2023 @05:00PM (#37884233)
In my country, drunk driving is punished by losing license and banning you from using another one for half year for first time and of life for second. And it's very effective, as those cases are rarity now
Re:All of ChatGPT's System Prompts by int_19h on Sunday October 15, 2023 @03:30AM (#37887715)
> It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice.

If they did, we'd be living in utopia already.

But also, by the same token, generative AI errors are similarly "corrected" by fine-tuning and RLHF. In both cases, you don't actually fix it - you just make it less likely to happen again.

Re:All of ChatGPT's System Prompts by manojlds on Saturday October 14, 2023 @09:03AM (#37880157)
Yeah, if we wanted to flap wings to fly, we would have never got the airplane.
Re:All of ChatGPT's System Prompts by Filligree on Saturday October 14, 2023 @11:02AM (#37880994)
The ornithopter is real, mind you.
Re:All of ChatGPT's System Prompts by stevenhuang on Saturday October 14, 2023 @09:08AM (#37880192)
> fundamentally flawed

Citation needed

Re:All of ChatGPT's System Prompts by Davidzheng on Saturday October 14, 2023 @10:33AM (#37880764)
Such a strange take. We have no choice but to use our brains??? It is also an incredibly capable system! At some point if the capabilities are so drastically different is it confusing that you would choose a much more capable system even with all its flaws?
Re:All of ChatGPT's System Prompts by pixl97 on Saturday October 14, 2023 @07:28PM (#37885258)
Because you are demanding and may rise up with pitchforks if the corporate class asks too much.

At the same time, humans are also unreliable as hell , push us much more than 8 hours a day of actual thinking work and our answers start to get crappy.

Re:All of ChatGPT's System Prompts by IshKebab on Saturday October 14, 2023 @11:48AM (#37881416)
Are you suggesting we just ignore the immense capabilities of these models because we don't fully understand them?

I hope you never need any medicine!

Re:All of ChatGPT's System Prompts by krainboltgreene on Saturday October 14, 2023 @05:00PM (#37884232)
> but you can say the same about human brains

It should be an HN rule that in order to type out variations of this sentence you have to also prove you have a degree in neuroscience.

Re:All of ChatGPT's System Prompts by xcdzvyn on Saturday October 14, 2023 @05:16PM (#37884345)
I think you should have to show your ML creds to claim LLM research is "pseudoscience", but here we are.
Re:All of ChatGPT's System Prompts by krainboltgreene on Sunday October 15, 2023 @03:16AM (#37887635)
Well it's not a science and you're going to have to a hard time convincing me that Richard Feynman is wrong.
Re:All of ChatGPT's System Prompts by int_19h on Sunday October 15, 2023 @03:29AM (#37887707)
You don't have to have a degree in neuroscience to be aware of the fact that we aren't even close to understanding how human brain works when it comes to high-level process such as sentience.
Re:All of ChatGPT's System Prompts by krainboltgreene on Sunday October 15, 2023 @08:36PM (#37894721)
First off we understand a lot about the human brain, but that doesn't matter because no one is arguing that we understand the totality of the human brain, instead what is being argued is: The human brain is as simple as an LLM and thus LLMs are sentient, can reason, and can know facts.
Re:All of ChatGPT's System Prompts by izacus on Saturday October 14, 2023 @08:56AM (#37880121)
Yeah, I wonder how much of the same could be achieved by just adding a few keywords.

Like in Ultima Online, where people could either do "Dear Sir, may I buy your wares?" which would be equivalent to "buy wares".

Re:All of ChatGPT's System Prompts by mentos on Sunday October 15, 2023 @07:34AM (#37888896)
vendor buy the bank guards
Re:All of ChatGPT's System Prompts by cornholio on Saturday October 14, 2023 @09:58AM (#37880520)
It's not pseudosience if the prompts are engineered according to the scientific method: formulate a hypothesis, experiment, reincorporate the results into your knowledge.

But it's a very fuzzy and soft science, almost on par with social sciences: your experimental results are not bounded by hard, unchanging physical reality; rather, you poke at a unknown and unknowable dynamic and self-reflexive system that in the most pathological cases behaves in an adversarial manner, trying to derail you or appropriating your own conclusions and changing its behavior to invalidate them. See, for example, economics as a science.

Re:All of ChatGPT's System Prompts by lgas on Saturday October 14, 2023 @02:58PM (#37883202)
> But it's a very fuzzy and soft science, almost on par with social sciences

There's no need for name-calling.

Re:All of ChatGPT's System Prompts by Davidzheng on Saturday October 14, 2023 @10:32AM (#37880749)
I think what you're asking for--this total scientific control and understanding of how the model will behave was never going to happen for any models close to our own intelligence.
Re:All of ChatGPT's System Prompts by rabbits_2002 on Sunday October 15, 2023 @01:58AM (#37887273)
I think thats a bit pessimistic. I am sure it would be possible to bake these prompts into the model but if this is equally effective and much less effort to do it this way, why is it an issue?
Re:All of ChatGPT's System Prompts by ChatGTP on Saturday October 14, 2023 @09:15AM (#37880231)
This isn't necessarily a uniform improvement, it is however, a great novelty at present. Time will tell.

I mean, yes it's cool I can write an essay to get Dall-E to generate almost the exact image I want (and I still can't using natural language), is it truly an improvement ? Yes I can churn out content faster, but I can't make the computer draw exactly what I want with words alone.

A picture is worth a thousand words, or maybe two thousand?

Re:All of ChatGPT's System Prompts by seydor on Saturday October 14, 2023 @10:30AM (#37880728)
It does something, not definitely the thing you asked
Re:All of ChatGPT's System Prompts by HaZeust on Saturday October 14, 2023 @06:45PM (#37884948)
It's even crazier to me that we've just... Accepted it, and are in the process of taking it for granted. This type of technology was a moonshot 2 years ago, and many experts didn't expect it in the lifetimes of ANYONE here - and who knew the answer was increasing transformers and iterating attention?

And golly, there are a LOT of nay-sayers of the industry. I've even heard some folks on podcasts and forums saying this will be as short-lived and as meaningless as NFTs. NFTs couldn't re-write my entire Python codebase into Go, NFTs weren't ever close to passing the bar or MCAT. This stuff is crazy!

Re:All of ChatGPT's System Prompts by PoignardAzur on Sunday October 15, 2023 @04:24AM (#37887979)
The crazy part is that it became banal so fast that people act like it's no big deal and will absolutely insist that the computer doesn't "actually" understand anything because it hasn't reached some perfectly interpretable platonic ideal of understanding yet, and anyone amazed that you can talk with the pile of sand is just being naive.