Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by disgruntledphd2 on Tuesday June 27, 2023 @07:09AM (#36491304)

Yes, but for most of these companies they don't actually run (much or any) spark stuff, they just use the notebooks.

It's really depressing, tbh but I've made my peace with it now.

[ Parent | on HN ]

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by pletnes on Tuesday June 27, 2023 @12:00PM (#36494697)

I can’t blame them really. Regular pandas-in-notebook can deal with quite a bit of data and is reasonably convenient. Even if the org and datasets are large, it’s rare to have huge single tables. Then you can go full spark when you have to.

[ Parent | on HN ]

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by disgruntledphd2 on Wednesday June 28, 2023 @03:52AM (#36503592)

Sure, but why bother paying databricks for that? By default you'll be running 3+ machines while doing all your work in memory on the master node. It's just so very, very silly.

[ Parent | on HN ]

Re:Databricks Strikes $1.3B Deal for Generative AI Startup M... by pletnes on Wednesday June 28, 2023 @09:57AM (#36506564)

True. I’ve been wondering why they don’t facilitate non-spark use better. But then you can’t reasonably invoice 3 VMs extra, obviously.

[ Parent | on HN ]