As for hosting, I found that runpod [2] has been the cheapest (not affiliated, just a user). All the other services tend to add up more than them when you include bandwidth and storage. There's some tutorials online [3] but a lot of them use the quantized version. You should be able to fit the original 70B with "load_in_8bit" on one A100 80GB.
[1] https://github.com/oobabooga/text-generation-webui [2] https://www.runpod.io/ [3] https://gpus.llm-utils.org/running-llama-2-on-runpod-with-oo...
Llama-2-70B is $1 / million tokens, which is the most cost-efficient on the market that I'm aware of.
This.
Google (specifically their CEO) was saying since at least 2016 that "Google is an AI first company". (Whatever the hell that means). But they had no product to show for and they are on the verge of being the next IBM.
Google is lagging behind in the market space for public AI tools, agree, but I am not convinced they are as far behind in AI development as you indicate.
What it means is it's why so many things about Google experience suck so badly. Whether it's what he meant or not, the practical flip side of Google being "AI first company" is that they're "humans last" company. Or, it's a different way of saying they only do things that scale. Telemetry and automated decision making scale. Human review and customer support do not.
They've released the most powerful open source LLM models so far (Llama 1 + 2) and are a serious threat to the Openai closed-source monopoly.
As promised, they released GPT3.5 fine-tuning today. They opened GPT4 API access a few months ago. In a few months, they'll release GPT4 fine-tuning.
Many favor open source AI, and criticize OpenAI for not being open enough. But the most important thing is, OpenAI innovates. Fast.
Llama, Bard, FB's open source stuff is good but it's lightyears behind OpenAI. You have to credit them for that.