News for hackers. Stuff that matters.
Re:The best way to use text embeddings portably is with Parq... by daveguy on Monday February 24, 2025 @03:39PM (#43164680)
Since a lot of ML data is stored as parquet, I found this to be a useful tidbit from lancedb's documentation:

> Data storage is columnar and is interoperable with other columnar formats (such as Parquet) via Arrow

https://lancedb.github.io/lancedb/concepts/data_management/

Edit: That said, I am personally a fan of parquet, arrow, and ibis. So many data wrangling options out there it's easy to get analysis paralysis.

There are no comments on this comment yet.