Oof, got access to the 8k model recently and was wondering what costs would be on the 32k one. That's brutal.

Why? 32K tokens would cover creating source code for a small to medium programming project. THat would easilly cost hundreds of dollars if done by a freelancer, and could get it in almost real time for just $2.

32k tokens is 500-1000 lines of code, so more like thousands of dollars, unless you're comparing against a landing page or CRUD tool arranged through Upwork. On the flip side, before I dove into AutoGPT I couldn't get GPT-4 to iterate on something as simple as a TypeScript function that deeply converts { foo_bar: string } to { fooBar: Date } without running out of context and cyclically reintroducing regressions.

Advice on where to get started with auto-gpt?

Only if you can get that result in one pass.

The real utility of LLMs is that they can be called in a loop to scan through many web pages, many code files, issue tickets, emails, etc...

There are already demos and experiments out there that for every input, 4x outputs are generated, then those are fed back into the LLM 4x for "review", then the best variant is then used to generate code which then automatically tested, errors are fed back in a loop, also with 4x parallel tries, etc...

It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.

> It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.

If its better in another dimension (e.g., calendar elapsed time), it may well be worth being more expensive.

Two dollars spent over and over would still take a while to equal a software engineer's salary, and if the end result is "produce a fully functional codebase in minutes" the premium might be worth it even if it exceeds that much.

We all know the game Sam Altman is playing. Eventually you can squeeze the full salary in cost out of companies who have laid off their workers and it would still be a good deal because AI does not need health care, pay taxes, hardware, sick days and so on.

Also, the azure offering is 10 times the openai price, so not really usable right now.

It's exactly the same pricing (East US):

https://azure.microsoft.com/en-us/pricing/details/cognitive-...

https://openai.com/pricing

Oh boy. I hope Microsoft doesn't get tired of subsidizing my access through Bing. I quite like it.

Bing seems to be using the stock 4K window size, which is likely why it's limited to that many messages in a single chat session before it forcibly shuts it down.

« other than cost »