Don't Pickle Your Data

Re:Don't Pickle Your Data by falcolas on Wednesday February 12, 2014 @04:10PM (#7227341)

JSON, while a fantastic lightweight data format, is insufficient for storing even moderately complex objects. It can not properly differentiate between tuples and lists, it can only accept strings for object keys, and it only stores unicode strings.

You also have to write custom (de)serializers if you want to store datetime objects (my personal pet peeve), any of the special python containers... basically any time you want anything other than an object, array, string or number.

Unless I'm dealing with untrusted data sources, or need to interoperate with other languages, I will keep using Pickle.

[ Parent | on HN ]

Re:Don't Pickle Your Data by drdaeman on Wednesday February 12, 2014 @05:14PM (#7227724)

Heh. Pickle is not really sufficient for some edge cases, too. Luckily, there's Dill that can take almost perfect snapshots of the whole interpreter: https://pypi.python.org/pypi/dill

[ Parent | on HN ]

Re:Don't Pickle Your Data by dman on Wednesday February 12, 2014 @06:54PM (#7228362)

Also Dill's developer Michael Mckerns is super responsive and helpful. Hopefully someday pickle will be replaced / augmented with dill in the standard distribution.

[ Parent | on HN ]

Re:Don't Pickle Your Data by boronine on Wednesday February 12, 2014 @04:47PM (#7227579)

I built an extensible type system on top of JSON for serialization and validation. Among other types, it ships with DateTime (it represents dates as ISO strings). It also lets you define your own types, as simple or complex as you need them to be. If you've got a minute to take a look, I would really appreciate some feedback :)

http://www.cosmic-api.com/docs/teleport/python/latest/

[ Parent | on HN ]

Re:Don't Pickle Your Data by brianpgordon on Wednesday February 12, 2014 @06:51PM (#7228340)

Also, most critically, JSON doesn't allow circular references.

[ Parent | on HN ]