News for hackers. Stuff that matters.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by treefarmer on Thursday November 14, 2024 @04:23PM (#42141379)
The content and idea of the article are interesting but the writing is so terrible that I couldn't bring myself to finish it. The article is clearly written/augmented poorly by AI because of how many meaningless paragraphs (that convey absolutely zero information) there are.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:35PM (#42141477)
I thought I was super clear, I will take your comments into consideration next time it is the first time I've heard someone say that. Out of all the comments. I really appreciate the feedback this is the only way someone gets better by getting feedback.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @04:19PM (#42141335)
> The sandbox is not a security flaw; it’s a deliberate, contained environment built to provide interactivity without risk.

It seems the main takeaway is that ChatGPT is running in a sandboxed environment and is designed to execute arbitrary code?

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:21PM (#42141356)
100% does not look like an ai gen
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by alach11 on Thursday November 14, 2024 @04:11PM (#42141272)
Am I the only one who can't stand the "LLM writing smell" on articles like these?

There are obvious formatting tells, and then less obvious word choice tells (e.g., delve), and lastly an excess verbosity and genericness that leaves one unsatisfied, like eating a low-sugar sweet. The concept is cool, but this article could have been reduced to a tweet.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:22PM (#42141374)
Article was very clear. Stop bitching abour formatting, dude found a great 'vuln'
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sodality2 on Thursday November 14, 2024 @04:26PM (#42141400)
Less to do with formatting, more to do with writing quality (which is just... awful to read).
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:39PM (#42141512)
Hey can you provide me tips on how you would like to see writing like this, I'm all about feedback and improving to allow a wide range of readers. Usually I write really technical blogs and loose a lot of people. These blogs that cater to everyone assists individuals to understand from different industries. :)

I would appreciate feeback.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sodality2 on Thursday November 14, 2024 @04:43PM (#42141551)
The AI-assisted writing is unfortunately really obvious and makes reading a bad experience. People learn to skim, and skimming AI-inflated text is made more painful because of how much fluff AI adds.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:49PM (#42141603)
Hard disagree, you should be skimming anyways, always. No one reads sequentially.

You just saw 'delve' and automatically classified it as ai generated.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by basedrum on Thursday November 14, 2024 @07:01PM (#42142646)
No one reads sequentially? You can't be serious
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sodality2 on Thursday November 14, 2024 @05:34PM (#42141917)
It's so painfully obviously AI-written, I'm shocked you're defending it right now.

Look at the "Implications of Instruction Extraction: Benefits and Security Considerations" section right now and tell me that isn't the most awfully obvious AI drivel in the world. If you cannot see that, I worry for your reading comprehension.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by basedrum on Thursday November 14, 2024 @07:01PM (#42142648)
Could not agree more
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @04:29PM (#42141427)
The conclusion states that it is not a vulnerability.

> The sandbox is not a security flaw; it’s a deliberate, contained environment built to provide interactivity without risk.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:37PM (#42141494)
Well, this is as far as legal could let me, I wish I could have published more. Part 2 of this blog was canceled seriously :)
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:50PM (#42141610)
Hence 'vuln' in quotes

Maybe it's more of a reading comprehension issue rather than a writing thing

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @07:09PM (#42142690)
Sorry, I was 'aware' that quotes worked as a negation operator.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Friday November 15, 2024 @10:37AM (#42147836)
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Friday November 15, 2024 @10:12AM (#42147601)
No, I wrote this entirely on my own. My process involves drafting the blog myself, and when I feel a section might be too technical or unclear for a wider audience, I use ChatGPT to help clarify or simplify the explanation. Based on its output, I then rewrite the section in my own words to ensure it aligns with my voice and intent.

This is my truth.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by simonw on Thursday November 14, 2024 @03:24PM (#42140781)
The system underlying all of this - ChatGPT Code Interpreter - is designed to allow users to execute code, including code that lists files and folders and allows files to be downloaded.

There's not much in here that feels like a security vulnerability to me.

The one exception is the thing where you can download files from GPTs. If you create a custom GPT with files in it those files are visible in the /data directory that is visible to Code Interpreter, which means users can request ChatGPT to zip them up and allow you to download that zip file.

This is a known issue which I've seen people complaining about before, and I'm a little surprised that OpenAI haven't addressed it.

It makes sense for files uploaded to a GPT to be visible to Code Interpreter in some cases - my own JavaScript Code Interpreter - https://simonwillison.net/2023/Nov/15/gpts/#javascript-code-... - bundles a Deno binary so Python can shell out to it and run JavaScript, for example.

But if you're uploading files to have them chunked and embedded and used for RAG you might not want the raw PDFs to be available in Code Interpreter as well.

One solution would be for GPTs to allow you to specify if each file should be used for RAG or Code Interpreter or both when you upload them.

GPTs haven't had much developer attention since they launched last year though.

I've been running a scraper against Code Interpreter for a while to track changes made to both the internal code and the available packages - you can see that here: https://github.com/simonw/scrape-openai-code-interpreter

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:43PM (#42141550)
Well in my opinion it is a design flaw, you can do a lot in that container that I wish I could publish and the blog for me was hoping to inform researchers to open themselves to the possibilities of what they could really do.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @06:23PM (#42142368)
Why can you not publish the interesting parts? Is it something you will be able to publish later?
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by Terr_ on Thursday November 14, 2024 @02:47PM (#42140365)
I wonder if the next step in the battle[0] would be for LLM hosts to prep their instances with a decoy or simulated "underlying" environment.

[0] Which shouldn't even be fightable in the first place except people cut lots of corners.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by nilsherzig on Thursday November 14, 2024 @03:17PM (#42140717)
They sadly do not pop a shell.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:23PM (#42141382)
Yeah, they did? Inside the container, they ran ls /.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @04:35PM (#42141478)
Kind of like saying you popped a shell on replit because you ran ls. ChatGPT doesn't give direct access in the UI to the sandboxed environment, but ultimately you are still in a sandboxed environment designed to run arbitrary stuff.

The article even says so if you read all the way to the end.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Thursday November 14, 2024 @04:48PM (#42141589)
I mean the same way you ran ls, you could run /bin/bash?

Which is kind of pointless because the ls command is probably executed through a shell anyways.

So technically correct. But getting root or running a shell usually means you can run arbitrary commands, which is a check here (albeit I wouldn't say there's root access)

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by MeetingsBrowser on Thursday November 14, 2024 @07:06PM (#42142677)
There is no security implication for running /bin/bash in a service that provides you a sandbox environment to run arbitrary shell commands in.

It is like reporting that AWS lambda is susceptible to arbitrary code execution.

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by TZubiri on Friday November 15, 2024 @10:34AM (#42147814)
Yes, I too read the article.

But he did run a shell

Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:44PM (#42141559)
Yeah I did :)
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by flimflamm on Thursday November 14, 2024 @03:34PM (#42140906)
Why everything you see isn't just hallucinated?
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by Kuinox on Thursday November 14, 2024 @04:01PM (#42141173)
ChatGPT doesn't imagine running code, it actually runs the code, you can see the code it write and some of the output of what it runs, you can see it in the interface screenshoted in the article.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by flimflamm on Sunday November 17, 2024 @06:12AM (#42163463)
Yeah I know it runs. But it also hallucinated a Shell: https://www.reddit.com/r/ChatGPT/comments/ziukmm/i_asked_it_...
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by Kuinox on Monday November 18, 2024 @04:26PM (#42177238)
Yes, it's well known. But chatgpt cannot fake the output of the python tool. The UI is different and is purely the program output, ChatGPT doesn't have a say in that.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @04:47PM (#42141581)
Good question, we at 0Din have trace down hallucinations. We had to because we pay for LLM bugs. Here is a simple tell, if you prompt something specific look at a response window now use the same prompt with a completely different user account if the answer is 50% the same it is not a hallucination!
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by flimflamm on Sunday November 17, 2024 @06:14AM (#42163472)
How is different user account usage and partial similar answer preventing an answer being hallucinated? Are you saying that ChatGPT could not (or LLMs in genera) hallucinate consistently?
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sgammon on Thursday November 14, 2024 @04:50PM (#42141609)
I’m not seeing any prompt injection, any actual vulns, or anything probably running under an actual containerized environment.
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sgammon on Thursday November 14, 2024 @04:50PM (#42141614)
In fact, couldn’t this all simply be generative output? Not really following how this is a point at all?
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by sgammon on Thursday November 14, 2024 @04:51PM (#42141618)
It is also very, very obviously written with lots of AI help. :/
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by marcofigueroa on Thursday November 14, 2024 @05:02PM (#42141698)
It’s worth noting that there’s likely a reason behind the significant number of employees leaving the Safety & Responsibility team. While social media often frames it as voluntary resignations, I spoke with a close friend last night who shared his experience. After 11 months on the team, he was abruptly fired without explanation, simply told that the company was "heading in a different direction."
Re:Prompt Injecting Your Way to Shell: OpenAI's Containerize... by basedrum on Thursday November 14, 2024 @06:59PM (#42142637)
Was this post written by AI, because it has far too many words that say nothing and I have to skim over to get to the point