News for hackers. Stuff that matters.
Re:Polars by YetAnotherNick on Monday January 08, 2024 @08:18PM (#38920986)
I don't know where this myth originated from but I have seen this in multiple places. Even if you think about it 2d numpy arrays can't have different type for different columns.
Re:Polars by billyjmc on Monday January 08, 2024 @08:51PM (#38921205)
Well, if you use structured arrays or record arrays, you can do this (more or less).

https://numpy.org/doc/stable/user/basics.rec.html

Re:Polars by theLiminator on Monday January 08, 2024 @08:37PM (#38921123)
Actually numpy has something called a structured array that is pretty much what what you described.
Re:Polars by FrustratedMonky on Monday January 08, 2024 @09:40PM (#38921589)
"myth originated"

It's in the documentation.

I just learned pandas recently, and I would have said this same thing. Not because I read through the numpy code, but because I read the documentation.

Is it wrong? Can't a user pick up a new tool and trust some documentation without reading through 100's of libraries built on libraries.

When was last time someone traced out every dependency, so they can confidently say something is "Myth".

Re:Polars by YetAnotherNick on Monday January 08, 2024 @10:44PM (#38922044)
Where is it written in pandas documentation? Pandas dataframe is stored in list of 1d numpy arrays, not a single 2d array.
Re:Polars by ayhanfuat on Tuesday January 09, 2024 @05:21AM (#38924419)
The columns with common dtypes are grouped together in something called "blocks" and inside those blocks are 2D numpy arrays. It is probably not in the documentation because it is seen as implementation detail but you can see the block manager's structure in this article (https://dkharazi.github.io/blog/blockmanager/) or in this talk (https://thomasjpfan.github.io/scipy-2020-lightning-talk-pand...).