How much does CoT Reasoning typically cost for frameworks like LlamaIndex?

Re:How much does CoT Reasoning typically cost for frameworks... by tikkun on Saturday September 14, 2024 @09:56AM (#41539834)

tough to answer. depends on the llm, on the api provider or hosting method, on the prompt

but you could get a simple estimate by running a CoT prompt, copying the output tokens, and then using an online token counter to count the length & tokens of the CoT part of the completion

[ Parent | on HN ]

Re:How much does CoT Reasoning typically cost for frameworks... by manili on Saturday September 14, 2024 @01:37PM (#41541381)

The issue is that I can give it a try on a set of toy problems, but finding a real-world-class problem is totally a different story IMHO. That is exactly the reason I asked about people's "experiences".

Anyway thanks for the response.

[ Parent | on HN ]

Re:How much does CoT Reasoning typically cost for frameworks... by kingkongjaffa on Saturday September 14, 2024 @02:09PM (#41541626)

> real-world-class problem

The issue is that this notion is nonsense. There are just problems nothing about them makes them “world class” whatever that means.

[ Parent | on HN ]

Re:How much does CoT Reasoning typically cost for frameworks... by manili on Sunday September 15, 2024 @12:53PM (#41548613)

Fair enough, it's "nonsense". Curious to know about YOUR experience. Sounds "nonsense" yet?

[ Parent | on HN ]