News for hackers. Stuff that matters.
Re:Ask HN: What's a side project you built to make money tha... by 37ef_ced3 on Wednesday December 30, 2020 @05:35PM (#25587055)
Addressing your last sentence, I am actually a GPU programmer by trade. GPUs are my specialty, and much of my professional work has been in Nvidia's CUDA

But AVX-512 is a real step forward, if you look at the price/performance ratio for real-world workloads. So AVX-512 is the focus of NN-512

GPU cloud compute is almost unbelievably expensive. Even Linode charges $1000 per month, or $1.50 per hour (look at the GPU plans: https://www.linode.com/pricing/#row--compute). It's really hard to keep that GPU saturated, which is what you need to do to get your money's worth

An AVX-512 Skylake-X cloud compute instance costs $10 per CPU-core per month, or 2 cents per hour, at Vultr (https://www.vultr.com/products/cloud-compute/). It's easy to saturate that core

Think about it this way: a GPU is really just a cluster of SIMD units with fast memory. A GPU thread variable is just like a SIMD lane, and a GPU warp variable is just like a SIMD vector register

Nvidia's SIMD pseudo-instructions are called PTX and they are like a 1024-bit version of AVX-512. An AVX-512 core is like a general purpose CPU with a 512-bit GPU core built in

So paying for a single AVX-512 core is like paying for part of a GPU (16 threads), plus the general purpose compute you need to keep that GPU part supplied with work

Re:Ask HN: What's a side project you built to make money tha... by bravura on Wednesday December 30, 2020 @07:02PM (#25587883)
I've been interested in your project. I don't think you're realizing the potential of the parent's suggestion.

You motivate the work through bottom-up construction, but I don't think you have a clear explanation for who the users are. They must be:

1) Sure they are only going to use models that are supported by NN-512. This is a big jump for me and many people who want to do transformers, LSTMs, etc. and not just what NN-512 costs. 2) They have to be price sensitive. But if you want larger companies, they will rarely just have one use-case that NN-512 supports; they will probably have a ton of use-cases that NN-512 supports. 3) They want to target AVX-512. If they are a medium or larger company they probably want to have tooling around this, to support model development and later deployment.

Those are my thoughts in general.

I currently need to be able to target and deploy on PC and Mac CPUs from the past 10 years. (I know, this is specific.)

I think parent is saying that individual devs are more likely to try NN-512 if they don't have to learn a new API, since most of them want to have the flexibility to fool around before committing to a new tech or bringing it in-house.

I guess my suggestion is really try to specifically define who the user is for NN-512 and what NN problems they are trying to solve, and let that shape the roadmap.

Re:Ask HN: What's a side project you built to make money tha... by 37ef_ced3 on Wednesday December 30, 2020 @07:26PM (#25588063)
My original thinking was that companies needing to use transformers, LSTMs, etc., would want to fund the development after doing the simple work to evaluate NN-512's ResNeXt or DenseNet performance on their hardware

Same with development of tooling (network conversion, parameter import). It just takes a lot of time/effort for me to write the code. It's easy to say that support for AVX, AVX2, etc., would be nice, but that's months of hard work (unfunded)

I guess we agree that a niche tool like NN-512, that has a reduced feature set (trimmed to allow one person to complete it and release it for free), has trouble competing with the huge, free, flexible, multi-target tools funded by large companies

In retrospect this is obvious!