News for hackers. Stuff that matters.
Re:Exploiting Misuse of Python's “Pickle” by Sir_Cmpwn on Sunday June 05, 2016 @06:21PM (#11843454)
I'd argue that any use of pickle is a misuse of pickle. Language-specific binary serialization formats should not exist, much less be used.
Re:Exploiting Misuse of Python's “Pickle” by Longhanks on Sunday June 05, 2016 @06:32PM (#11843493)
While designed for Python, pickle is not language specific, there are implementations for .NET and Java that work well, for example: https://github.com/irmen/Pyrolite
Re:Exploiting Misuse of Python's “Pickle” by xapata on Sunday June 05, 2016 @11:23PM (#11844463)
What do you think of ``eval``?

I'll go ahead and keep using pickle. It makes me more productive, despite never unpickling untrusted data.

By the way, reading XML is a security risk, too...

Re:Exploiting Misuse of Python's “Pickle” by GFK_of_xmaspast on Monday June 06, 2016 @11:32AM (#11847515)
> What do you think of ``eval``?

Never ever.

Re:Exploiting Misuse of Python's “Pickle” by xapata on Monday June 06, 2016 @01:20PM (#11848431)
I guess you can never make a REPL then.

Or namedtuple, one of my favorite tools.

Re:Exploiting Misuse of Python's “Pickle” by filmor on Monday June 06, 2016 @03:52AM (#11845249)
Pickle is used in Python's multiprocessing library to communicate with other Python processes over a pipe. Seems sensible to me.
Re:Exploiting Misuse of Python's “Pickle” by fulafel on Monday June 06, 2016 @04:53AM (#11845425)
The default pickle format is not binary, though it's about as human-readable as sendmail.cf:

  >>> cPickle.dump(dict(foo=42), sys.stdout)
  (dp1
  S'foo'
  p2
  I42
  s.
Re:Exploiting Misuse of Python's “Pickle” by quasiben on Sunday June 05, 2016 @07:34PM (#11843727)
What should I rely on if I want to serialize a function ?
Re:Exploiting Misuse of Python's “Pickle” by GauntletWizard on Sunday June 05, 2016 @08:31PM (#11843923)
Mu. Unask the question. Serializing a function is itself the security hole, because who knows what that function does? A function is just as possibly malicious code as whatever else you think you're using it for.
Re:Exploiting Misuse of Python's “Pickle” by xapata on Sunday June 05, 2016 @11:19PM (#11844447)
Being connected to the internet is a security hole. I'll balance security with practicality.
Re:Exploiting Misuse of Python's “Pickle” by ludamad on Monday June 06, 2016 @02:21AM (#11844998)
The GP's point is if you're doing code deserialization, the definition of security is different. The data format having RCE bugs won't be as much of a concern, while trusting the data source will be much more of a concern.
Re:Exploiting Misuse of Python's “Pickle” by xapata on Monday June 06, 2016 @01:22PM (#11848446)
We redefine security for every project. Some projects can (de)serialize code and be secure. Others can't.
Re:Exploiting Misuse of Python's “Pickle” by IanCal on Monday June 06, 2016 @02:24AM (#11845011)
If someone is able to modify my local files and wants to change the code I'm running, what is it about pickle that makes me vulnerable? They'd already be in a position where they could change my actual code.
Re:Exploiting Misuse of Python's “Pickle” by GauntletWizard on Monday June 06, 2016 @02:53AM (#11845085)
Why do you think the attacker has the ability to change your local files? That's not the attack surface. The attack surface is when it's used in a wire format for network communication. The attack surface is the attacker convincing you to download a file in a format that you don't realize is executable and every file format that includes pickled objects is vulnerable. The attack surface is that you've confused data and code; There's a reason why OpenBSD now enforces W^X, and that is that there's very different domains of trust. I'd trust my neighbor down the street to send me a spreadsheet, I'd not trust him to send me a program (which is precisely why Office Macro viruses were such a problem, because speadsheets could contain programs.)

No, pickling and unpickling files in your user's data-dir isn't a big deal, until it is because your users wanted to share their data online. They'll sync their data-folder with Dropbox, and then their account gets compromised, and the next time they launch your program they've got a virus. They'll download a 'completed savegame' from a sketchy site, and now they've got a virus. They'll get a phishing e-mail, but it's not one of those zips or exes or whatever, it's a file-format they know is your program, and surely that's safe, right?

Don't make your file formats insecure. Don't pickle.

Re:Exploiting Misuse of Python's “Pickle” by IanCal on Monday June 06, 2016 @04:42AM (#11845397)
My rather poorly made point is that whether it's a security problem or not depends on what you're using pickle for. My typical use case is simply as a local cache for some slow computation in ad-hoc scripts. Being able to simply dump objects and load them back again without needing to write serialisation code is a great timesaver, and the only ways I can see that causing a security problem rely on an attacker already having significantly more access to begin with.

I don't like people making such strong statements about what others should and should not do, based on issues in some situations.

Re:Exploiting Misuse of Python's “Pickle” by ben_jones on Monday June 06, 2016 @03:09AM (#11845132)
You can call functions across processes by using RPC libraries.
Re:Exploiting Misuse of Python's “Pickle” by orf on Monday June 06, 2016 @05:21AM (#11845499)
Cool, but not even slightly relevant.
Re:Exploiting Misuse of Python's “Pickle” by ank_the_elder on Sunday June 05, 2016 @07:50PM (#11843785)
Is your criticism about the fact that pickle is language-specific or about it being a binary format?
Re:Exploiting Misuse of Python's “Pickle” by philsnow on Sunday June 05, 2016 @08:00PM (#11843814)
(not the person you're replying to, but) my criticism of python pickle is that it allows the deserialized thing to refer to objects outside of a restricted environment (in this case, subprocess.Popen). JSON deserialization can't refer to console.log or any other object.

Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything. (Even then, it's still potentially dangerous, because in a large project some other dev who doesn't understand the issue could come along and write a new thing to deserialize the same pickles and not check the HMAC).

Re:Exploiting Misuse of Python's “Pickle” by IanCal on Monday June 06, 2016 @02:22AM (#11845005)
> Unless you're storing an HMAC alongside your pickled blob, and verifying that HMAC on the deserialization side, you should not trust pickle for anything.

I dislike such strong statements. When I use pickle, it's often a dump of where a computation is up to, or a simple cache. How can that be exploited? Someone would need to be meddling on my filesystem, in which case I'm already screwed. What risks am I opening myself up to? Why should I not trust it for my use case?

Re:Exploiting Misuse of Python's “Pickle” by philsnow on Monday June 06, 2016 @07:44PM (#11851403)
It depends on the kind of groups/organizations you write code in. In some, I think such strong statements are necessary.

You can mitigate the dangers of pickle by educating junior devs or people new to python, by having a strong culture of code reviews with experienced reviewers, by having a style guide that explicitly prohibits pickle except in exceptional cases.

... Or you can just not use it in the first place and make it trivial for newcomers to your codebase to use safer serialization methods.

Re:Exploiting Misuse of Python's “Pickle” by vertex-four on Monday June 06, 2016 @02:54AM (#11845090)
Well, specifically, never use pickle for communication. It's perfectly reasonable as an on-disk storage format for some things.
Re:Exploiting Misuse of Python's “Pickle” by sitkack on Sunday June 05, 2016 @10:10PM (#11844223)
If you want a "secure pickle" you should corrupt it on creation so that it needs a to uncorrupted to be read. preferably with the HMAC.
Re:Exploiting Misuse of Python's “Pickle” by ashitlerferad on Monday June 06, 2016 @12:46AM (#11844719)
No, just switch from pickle to json.
Re:Exploiting Misuse of Python's “Pickle” by sitkack on Monday June 06, 2016 @01:23AM (#11844830)
But how would you get arbitrary code execution? Pickle does solve a lot of problems, but it creates so many that I think it should be removed from the battery pack. You can't make HMAC integrity checks optional, and you make the mandatory by breaking the payload.
Re:Exploiting Misuse of Python's “Pickle” by sametmax on Monday June 06, 2016 @04:56AM (#11845431)
I use it save my shell session between reboots. Very useful, yet no security concern. There is always a niche for any tech.
Re:Exploiting Misuse of Python's “Pickle” by gaius on Monday June 06, 2016 @05:53AM (#11845590)
Interesting. How are you doing it? Do you do objects individually?
Re:Exploiting Misuse of Python's “Pickle” by sametmax on Monday June 06, 2016 @06:00AM (#11845603)
Yes. Basically I got a python startup file opening a store object at the begining of the session. If you set an attribute to the object, it's saved in the store, and if you read it, it loads it.
Re:Exploiting Misuse of Python's “Pickle” by gaius on Monday June 06, 2016 @06:04AM (#11845615)
Cool. I'd love a way to completely persist a session to disk and reinstate it later, tho'. In other languages too http://stackoverflow.com/q/3966925/447514 - but only languages like Smalltalk or Forth with the concept of an "image" really do what I want.
Re:Exploiting Misuse of Python's “Pickle” by fulafel on Monday June 06, 2016 @09:52AM (#11846729)
OS level support for checkpointing is also a thing that exists: Eg https://criu.org/
Re:Exploiting Misuse of Python's “Pickle” by sametmax on Monday June 06, 2016 @06:15AM (#11845658)
I think we will come to that in the future, but the OS will handle that instead of the language.

Of course, one can argue you can already to it with docker :)

Re:Exploiting Misuse of Python's “Pickle” by PeCaN on Monday June 06, 2016 @04:31PM (#11850073)
The future as in, say, 1979? https://en.wikipedia.org/wiki/System/38
Re:Exploiting Misuse of Python's “Pickle” by gaius on Monday June 06, 2016 @06:24AM (#11845685)
Well you could do it for decades with VMs but that's pretty heavyweight ;-)
Re:Exploiting Misuse of Python's “Pickle” by EwanToo on Monday June 06, 2016 @01:59AM (#11844930)
If you want a more efficient version of pickle, take a look at feather [1]

It's not a drop in replacement but it's very effective

1 - https://github.com/wesm/feather