Re:Does current AI represent a dead end?
by sroussey on Friday December 27, 2024 @03:38PM
(#42525820)
They always return the same output for the same input. That is how tests are done for llama.cpp, for example.
To get variety, you give each person a different seed. That way each user gets consistent answers but different than each other. You can add some randomness in each call if you don’t want the same person getting the same output for the same input.
It would be impossible to test and benchmark llama.cpp et al otherwise!
By the time you get to a UI someone has made these decisions for you.
To get variety, you give each person a different seed. That way each user gets consistent answers but different than each other. You can add some randomness in each call if you don’t want the same person getting the same output for the same input.
It would be impossible to test and benchmark llama.cpp et al otherwise!
By the time you get to a UI someone has made these decisions for you.
It’s just math underneath!
Hope this helps.