News for hackers. Stuff that matters.
Re:Ursa Labs: an innovation lab for open source data science by neves on Thursday April 19, 2018 @02:30PM (#16878663)
Just to be clear, the author is Python Pandas creator and main maintainer. He knows what he is talking about. These are great news.

Hope they get a lot of funding.

Re:Ursa Labs: an innovation lab for open source data science by wesm on Thursday April 19, 2018 @02:53PM (#16878896)
I'm not the main maintainer anymore (Jeff Reback has that honor), as I've moved on to work on the "greater pandas ecosystem" (of which my work on Apache Arrow is a part of this long development arc)
Re:Ursa Labs: an innovation lab for open source data science by skadamat on Thursday April 19, 2018 @03:28PM (#16879238)
Super excited. Just reached out to the email on the site on how I can help technically in the future!
Re:Ursa Labs: an innovation lab for open source data science by saltandvinegar on Thursday April 19, 2018 @08:48PM (#16881354)
It sounds like with RStudio and Two Sigma they've got startup capital at least. I hope others join them in funding this important work!
Re:Ursa Labs: an innovation lab for open source data science by psychometry on Thursday April 19, 2018 @03:31PM (#16879267)
I'm a little confused about what the product is. They are using Apache Arrow to build...something. How would this stuff benefit me as an R programmer?
Re:Ursa Labs: an innovation lab for open source data science by makmanalp on Thursday April 19, 2018 @03:44PM (#16879407)
It would change things primarily under the hood for table-like data structures ... imagine having a data.frame or tibble object but in memory it's stored in a very specific layout which allows it to a) make use of pre-created highly optimized library code to do complicated queries b) be interchanged with other programs with almost zero overhead, e.g. imagine doing a query on a massive database in spark and then loading it into R or pandas instantly without having to wait.
Re:Ursa Labs: an innovation lab for open source data science by psychometry on Thursday April 19, 2018 @04:35PM (#16879897)
Thanks. Do you think there is a use case for traditional "small data" analysis in R, i.e. standard CSV->dplyr->output sort of transformations?
Re:Ursa Labs: an innovation lab for open source data science by hadley on Thursday April 19, 2018 @04:42PM (#16879965)
It should make things a bit faster, and a bit easier to collaborate with people using other languages.
Re:Ursa Labs: an innovation lab for open source data science by WhompingWindows on Thursday April 19, 2018 @04:18PM (#16879729)
If the author comes in here, let me say: thank you for Feather. I am currently using that for a project on very slow servers and it is helping me quite a bit.

My question here is this: like the creator of Vue.JS has done, could Ursa Labs go the way of Patreon, and have potential personal, academic, or corporate clients simply donate to ensure contributed support and production of great tools for data science?

Re:Ursa Labs: an innovation lab for open source data science by makmanalp on Thursday April 19, 2018 @03:16PM (#16879120)
Very happy to see Apache Arrow gaining support! It'd be very cool to see modern columnar storage architecture gain widespread adoption: there's decades of research in it and massive improvements for analytical workloads.
Re:Ursa Labs: an innovation lab for open source data science by tmandry on Thursday April 19, 2018 @02:22PM (#16878567)
I'm really happy to hear this. The vision for Arrow is solid, and I'm looking forward to a future where C++, Python, and other languages like Rust can interoperate more smoothly in an integrated analysis workflow. I hope Wes and Ursa Labs will be a major factor in developing the way we approach data science over the next 10+ years.
Re:Ursa Labs: an innovation lab for open source data science by vtuulos on Thursday April 19, 2018 @07:42PM (#16881116)
This is very exciting, thanks Wes! A robust, high-performance, polyglot, in-memory runtime for data(frames) would be extremely useful.

The remarks about challenges related to OSS maintenance/innovation were spot on too.