Yea the sql queries, string or array manipulation are better and fast than google 8/10 times because of the errors it generates.
Granted, if you are spending 10x the future value of the product your are offering ... then even a 10x decline in compute costs won't get you where you need to be.
they quantized model from 16 bits to 4 bits which was low hanging fruit, and looks like they can't quantize it anymore to 2 bits..
Pixel 6+ phones have TPUs (in addition to CPUs and an iGPU/dGPU).
Tensor Processing Unit > Products > Google Tensor https://en.wikipedia.org/wiki/Tensor_Processing_Unit
TensorFlow lite; tflite: https://www.tensorflow.org/lite
From https://github.com/hollance/neural-engine :
> The Apple Neural Engine (or ANE) is a type of NPU, which stands for Neural Processing Unit.
From https://github.com/basicmi/AI-Chip :
> A list of ICs and IPs for AI, Machine Learning and Deep Learning
In my opinion, Microsoft's proposed 365 Co-pilot pricetag of $30 per month per user will probably make a profit. High usage users being offset by low usage users when corporate 365 group memberships come into the mix. Many corporates will take a chance on it, willing to throw plenty of money at anything that is perceived as having a chance of improving hard to define competitive edge, creativity and productivity.
I think we've seen some people striving for a competitive advantage, and willing to see one exist, but I don't know if we've seen any "AI company" turn a meaningful and durable profit yet (open to being wrong on this).
To me (so far), it seems like GPT use cases that apply it as a table stakes feature of some greater application (rather than an ecosystem or a platform), are the ones that are actually showing promise in terms of expected utility + value capture.
If GPT4+ level engines become as cheap to execute as a SQLite query, I think that's where things get interesting (you can start executing this stuff at the edge).
But I still can't see new companies (a la OpenAI, MidJourney, etc.) making a lot of money in this scenario, it seems to overwhelmingly favor companies that already have distribution.