News for hackers. Stuff that matters.

A comment on Show HN: Kim – A Python serialization and marshaling framework. Parent comment

Re:Show HN: Kim – A Python serialization and marshaling fram... by fnord123 on Wednesday April 12, 2017 @10:55AM (#14098234)
McKinney has been hard at work getting parquet and arrow support in pandas.

http://wesmckinney.com/blog/outlook-for-2017/

>Give the kind of people behind Arrow, I would love wrapper that will use Arrow to do all of this...But doesn't matter at the end of the day.

pyarrow; pyarrow.parquet (which uses parquet-cpp).

[ Parent | on HN ]
Re:Show HN: Kim – A Python serialization and marshaling fram... by dtjon on Wednesday April 12, 2017 @12:32PM (#14099269)
Wow this is great. I've been working around the jvm to integrate sklearn and some spark jobs that produce Parquet. This is a huge relief
[ Parent | on HN ]
Re:Show HN: Kim – A Python serialization and marshaling fram... by sandGorgon on Wednesday April 12, 2017 @12:45PM (#14099378)
Arrow doesn't do scikit - atleast last time I checked . Has it changed ?
[ Parent | on HN ]
Re:Show HN: Kim – A Python serialization and marshaling fram... by dtjon on Wednesday April 12, 2017 @12:47PM (#14099410)
pyarrow has methods to convert to pandas, which scikit supports

http://pyarrow.readthedocs.io/en/latest/pandas.html

[ Parent | on HN ]
Re:Show HN: Kim – A Python serialization and marshaling fram... by sandGorgon on Wednesday April 12, 2017 @04:37PM (#14101504)
No - this is not it. Scikit models need to be persisted. The only ways I have found is pickle or dill.

Take a look at this to understand what I mean . http://stackoverflow.com/questions/32757656/what-are-the-pit...

[ Parent | on HN ]