News for hackers. Stuff that matters.
Re:When AWS, Azure, or GCP Becomes the Competition by m_ke on Monday October 28, 2019 @03:56PM (#21380935)
If you're doing anything ML related your best bet is to niche down and solve a very narrow problem for a specific industry / market segment. Building generic AI tooling sounds great but rarely solves anyones problems.
Re:When AWS, Azure, or GCP Becomes the Competition by mlthoughts2018 on Monday October 28, 2019 @07:52PM (#21383033)
As an ML practitioner in a large ecommerce company, I totally agree. When I evaluate third party tools, 99% have no use at all for me. Things like Algolia or Rekognition or Clarifai.

Even Sagemaker and Fargate are mostly not useful to me as my company can easily operate its own k8s platform.

Open source tools for model building are great and a small team of engineers can write whatever odds & ends that aren’t covered, or make scalable implementations.

Notebook platforms like Databricks are similarly useless. A notebook is not a frontend to a cluster and notebooks are universally poor development environments even for the very tasks they are advertised.

The main thing that matters for ML development is how quickly & easily you can put an arbitrary container or VM into a deployment environment. You’re always going to need to rapidly change your container or VM, have common ancestors that get specialized to support GPU training or some web server application layer. Making the shortest possible distance between the definition of this environment and deployment of it is the whole game.

The type of tools I’m willing to pay for are specialized database engines, rapid data annotation tools, and specialized hardware environments.

I have no time for some “platform” tool for data provenance, notebook environments, model tracking, model-as-a-service APIs like Rekognition, or anything where I upload data and get back a model.

Re:When AWS, Azure, or GCP Becomes the Competition by tixocloud on Tuesday October 29, 2019 @05:25AM (#21385327)
Could you care to elaborate further on specialized database engines and rapid data annotation tools? There are plenty of data annotation tools out there that seem to do the job so not sure what rapid would mean in this instance.
Re:When AWS, Azure, or GCP Becomes the Competition by mlthoughts2018 on Tuesday October 29, 2019 @06:55AM (#21385712)
I mean things like vertica or kdb+ that have specialized performance properties for some use cases. Also to some minor extent managed cluster pipeline tooling like Spark. I don’t mind paying for managed versions of these (not Databricks though). For annotation tools I mean things like Prodigy.
Re:When AWS, Azure, or GCP Becomes the Competition by tixocloud on Tuesday October 29, 2019 @08:52AM (#21386528)
Thanks. Why the interest in managed cluster pipeline but not Databricks? They seem to be the big name in the game for Spark. Would some tooling around snapshotting, sequential AB testing, staged rollouts, ghost models be of use?
Re:When AWS, Azure, or GCP Becomes the Competition by mlthoughts2018 on Tuesday October 29, 2019 @09:50AM (#21387110)
Databricks is a notebook frontend to cluster computing. Cluster computing is useful and I’m willing to pay for it as long as I have total & complete control of the environments and tooling used in the cluster workflow.

A notebook frontend however is less than useless, and is actively harmful by propagating poor notebook environments even further into aspects of computing where they cause harm and hurt reproducability and code factoring.

Given this, even if Databricks offered perfectly complete features for all aspects of cluster computing, it would still be inferior to just my own managed EC2 or EMR clusters or equivalent with other providers, where there is no “notebook as control plane” garbage.

But when you add to that the fact that Databricks lacks full features for me to totally own every customized detail of my cluster environment (e.g. how can I run plain Python multiprocessing tasks with zero Spark in Databricks? How can I bring my own custom defined GPU container with custom compiled Tensorflow?) it makes the deal even worse.

Databricks is just another Spark / Hadoop style snake oil seller banking on capturing a bunch of data science teams before people realize that it’s a conceptually junk way to work.

As for the other tooling you mention, I’d almost always say to build it in house. For example, I don’t know of a single A/B testing provider that actually uses frequentist sequential testing to correctly avoid early stopping bias. You actually need real statisticians hired in-house to solve these problems, and the engineering work to set up an extensible A/B test as a service internal tool is not bad. (I’ve built Bayesian A/B test frameworks with teams of 2-3 engineers in 3 different companies). It’s just not cost effective to outsource it on the false hopes of not needing to hire your own in-house statistics experts. Just pony up the dough and hire them.

Re:When AWS, Azure, or GCP Becomes the Competition by mistrial9 on Monday October 28, 2019 @10:49PM (#21383900)
> notebooks are universally poor development environments even for the very tasks they are advertised

unnecessarily dismissive, grandstanding.. Bravado of the rejection, not supported at all

Rich-content Notebooks tell a story; well-worn Notebooks are a familiar context for experiment and viz.

Re:When AWS, Azure, or GCP Becomes the Competition by FridgeSeal on Monday October 28, 2019 @11:17PM (#21384013)
Notebooks are a maintainability maze waiting to happen.

They're incredibly non-portable; they don't play nice with version control; I know how they're supposed to be used, but I'm yet to see them actually used in a way that isn't a breeding ground for the worst software engineering I've seen.

Re:When AWS, Azure, or GCP Becomes the Competition by un_montagnard on Tuesday October 29, 2019 @01:35AM (#21384468)
Yet the same could be said of Excel, which is running the whole financial industry.
Re:When AWS, Azure, or GCP Becomes the Competition by gpderetta on Tuesday October 29, 2019 @05:40AM (#21385396)
It can be both a bad environment and very successful.

Personally I would be happy with just proper version control support.

Re:When AWS, Azure, or GCP Becomes the Competition by ordx on Monday October 28, 2019 @09:52PM (#21383631)
How do you train your models? With Databricks (which is just a nice notebook UI for Apache Spark) I don't have to spend time setting up VMs and libraries for the distributed training.
Re:When AWS, Azure, or GCP Becomes the Competition by mlthoughts2018 on Tuesday October 29, 2019 @07:05AM (#21385772)
Your first sentence seems unrelated to your second sentence, but it seems like you think they are related?

Any platform that advertises something like you don’t have to spend time defining your training environment, whether it’s Databricks or an out of the box deep learning VM on GCP, is a liability waiting to happen. You always need to define your own training environment, especially because you’ll almost always need specific (likely pinned) versions of all your dependencies, including system dependencies, to manage model training as part of a production life cycle. Very often you also need e.g. custom compiled TensorFlow, custom GPU settings or drivers, etc. It’s very foolish to base that environment on whatever comes out of the box.

Spark also is not universally useful. For training small models many times, like a workload that trains hundreds of small models all day (this was a production use case I had before that my company pilot tested with Databricks), the overhead of Py4J connector is insanely bad. It’s a really terrible paradigm for Python software, meanwhile Scala is a miserable ecosystem for production machine learning models. On top of all this, Spark MLib has huge gaps in functionality and whole classes of problems (e.g. large scale MCMC inference) are not solvable in a way in Spark that is seriously comparable to other tools like STAN & pymc running on not-Spark with simple multiprocessing.

Re:When AWS, Azure, or GCP Becomes the Competition by ordx on Tuesday October 29, 2019 @08:06PM (#21393426)
Interesting, sounds like you have a very specific usecase. I'm mostly dealing with huge datasets and Spark is a lifesaver.
Re:When AWS, Azure, or GCP Becomes the Competition by mlthoughts2018 on Wednesday October 30, 2019 @07:14AM (#21396408)
My use case is just general computing on huge datasets for ML and statistics.

Spark only serves a few special use cases that usually don’t justify its cost, similar to map reduce on Hadoop.

Databricks though (distinct from Spark) serves no use cases.

Re:When AWS, Azure, or GCP Becomes the Competition by gk1 on Monday October 28, 2019 @04:29PM (#21381302)
Agreed. One of the possible differentiators I list in the article is being an "end-to-end solution." It's going to be a long while before AWS builds AI for accounting, or for optimizing industrial operations, or for... you get it.
Re:When AWS, Azure, or GCP Becomes the Competition by generatorguy on Monday October 28, 2019 @05:36PM (#21381986)
I have yet to hear of an example of a factory or plant using AI for optimizing industrial operations. Anybody known of any?
Re:When AWS, Azure, or GCP Becomes the Competition by mttddd on Monday October 28, 2019 @05:55PM (#21382143)
I imagine there are some, a lot of factory operation optimization uses heavy operations research which isnt a huge jump to some sort of AI/ML
Re:When AWS, Azure, or GCP Becomes the Competition by digsy on Monday October 28, 2019 @06:21PM (#21382366)
Re:When AWS, Azure, or GCP Becomes the Competition by lostmsu on Monday October 28, 2019 @06:57PM (#21382656)
Interesting, as I went down this route: made a niche ML product, and did not open source. As it is a framework-level library, I also think quite a bit of time will pass before AWS or Azure will even notice the opportunity, because you need to be technical to see it.

Actually, Azure already has ML.NET, but it is about classical machine learning, and their own deep learning product, CNTK, lost the marketing fight with TensorFlow and PyTorch. Which is where I come in :)