GPT-4-turbo preliminary benchmark results on code-editing

Re:GPT-4-turbo preliminary benchmark results on code-editing by andy_xor_andrew on Tuesday November 07, 2023 @07:03PM (#38184943)

I think the general (completely speculative and unconfirmed) consensus is that "Turbo" models are somehow quantized, or otherwise modified for much faster and cheaper inference, at the expense of some quality (the definition of "some" is unknown here).

well, that's how it was for GPT-3.5 anyway. The "turbo" flavor was faster and cheaper, but seemed to have slightly worse output (again, this is all going by subjective measurement; it could entirely be the imagination of AI bros)

[ Parent | on HN ]

Re:GPT-4-turbo preliminary benchmark results on code-editing by xeckr on Tuesday November 07, 2023 @07:07PM (#38184982)

I agree that this is the case for GPT-3.5, at least subjectively. However, with GPT-4 Turbo, it seems that performance has improved. If they got it to be faster using quantization, then they must have also found a way to offset any resulting performance losses.

[ Parent | on HN ]

Re:GPT-4-turbo preliminary benchmark results on code-editing by letitgo12345 on Tuesday November 07, 2023 @08:19PM (#38185526)

They have tons of usage data by now to figure out which queries to devote model capacity to

[ Parent | on HN ]