1. Improve the web site slightly (add a contact method and pricing info) and make it clear that it's a company offering an enterprise software tool. Programmers will be viewing the site, so it doesn't need much more than is there already.
2. Charge a flat $7,500/year for a commercial usage license and increase it as the functionality increases. Offer email support and updates. Many programmers can buy a $7,500/year piece of software without their management even asking a follow up question.
3. Write some blog posts showing how to use it and why it's good. Some benchmarks (LOC, performance, etc) vs alternatives would be a good source of posts. Real world examples that people can copy/paste to start with is probably the most useful.
4. Submit the blog posts to HN and some subreddits.
5. Profit (maybe)
I’d like to know where you work.
No place I’ve ever worked would approve that without a lot of questions.
My suggestion would be to focus on an initial use case where a very limited low cost / high efficiency CPU model can provide massive advantage. NN-512 should be the framework that expands from that Redis like core. The limited use case tactic is what I'm focusing on[1], mainly as I have a particular application and have less technical brilliance than yourself so need to focus ;)
An aged but still relevant example is the early word2vec work which was (and still is) frequently better to throw onto CPUs than GPUs. A well tuned implementation is not only advantageous on CPU but can win out in many scenarios where cost / latency / ... are important.
Congrats on the project though! I'd be curious for your thoughts for the future if you ever want to chat =]
[1]: Initial experiments written up as a tutorial with Rust and ISPC for a specific CPU based NN task - https://state.smerity.com/smerity/state/01E8RNH7HRRJT2A63NSX...
But AVX-512 is a real step forward, if you look at the price/performance ratio for real-world workloads. So AVX-512 is the focus of NN-512
GPU cloud compute is almost unbelievably expensive. Even Linode charges $1000 per month, or $1.50 per hour (look at the GPU plans: https://www.linode.com/pricing/#row--compute). It's really hard to keep that GPU saturated, which is what you need to do to get your money's worth
An AVX-512 Skylake-X cloud compute instance costs $10 per CPU-core per month, or 2 cents per hour, at Vultr (https://www.vultr.com/products/cloud-compute/). It's easy to saturate that core
Think about it this way: a GPU is really just a cluster of SIMD units with fast memory. A GPU thread variable is just like a SIMD lane, and a GPU warp variable is just like a SIMD vector register
Nvidia's SIMD pseudo-instructions are called PTX and they are like a 1024-bit version of AVX-512. An AVX-512 core is like a general purpose CPU with a 512-bit GPU core built in
So paying for a single AVX-512 core is like paying for part of a GPU (16 threads), plus the general purpose compute you need to keep that GPU part supplied with work
You motivate the work through bottom-up construction, but I don't think you have a clear explanation for who the users are. They must be:
1) Sure they are only going to use models that are supported by NN-512. This is a big jump for me and many people who want to do transformers, LSTMs, etc. and not just what NN-512 costs. 2) They have to be price sensitive. But if you want larger companies, they will rarely just have one use-case that NN-512 supports; they will probably have a ton of use-cases that NN-512 supports. 3) They want to target AVX-512. If they are a medium or larger company they probably want to have tooling around this, to support model development and later deployment.
Those are my thoughts in general.
I currently need to be able to target and deploy on PC and Mac CPUs from the past 10 years. (I know, this is specific.)
I think parent is saying that individual devs are more likely to try NN-512 if they don't have to learn a new API, since most of them want to have the flexibility to fool around before committing to a new tech or bringing it in-house.
I guess my suggestion is really try to specifically define who the user is for NN-512 and what NN problems they are trying to solve, and let that shape the roadmap.
Same with development of tooling (network conversion, parameter import). It just takes a lot of time/effort for me to write the code. It's easy to say that support for AVX, AVX2, etc., would be nice, but that's months of hard work (unfunded)
I guess we agree that a niche tool like NN-512, that has a reduced feature set (trimmed to allow one person to complete it and release it for free), has trouble competing with the huge, free, flexible, multi-target tools funded by large companies
In retrospect this is obvious!
Tell me if you can find one
The idea was that user companies (or even the hardware companies: Intel, AMD) would pay me as a contractor to implement deconvolutions, transformers, etc., as extensions to the free/open core I released (which includes very sophisticated support for all forms of dense convolution, fully-connected layers, pooling, batch norm, activations, etc.)
The free/open core supplies all the basic operators, but many users would need extensions of the core to support their particular networks (or even just software to feed their parameters into the system and get results out) and hardware companies benefit from the software's existence because they sell chips and NN-512 makes the new vector units on their chips more useful
But it didn't work out that way. NN-512 did generate offers to work on other projects (a generous offer from Intel, for example) but nothing that extends NN-512
Why? There are numerous free end-to-end deep learning tools developed by large, company-sponsored teams (Nvidia, Facebook, Google, Intel, etc.) so a specialized tool that requires integration effort is unattractive. That's my guess. So beware, if you're working along similar lines