News for hackers. Stuff that matters.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by danielhanchen on Tuesday January 28, 2025 @04:15PM (#42858119)
Ye lower temp is also good :) Tbh its all trial and error - I found temp=1.5, min_p=0.1 to be very useful for pass@k type workloads - ie calling the LLM multiple times and aggregating.

temp=0 is also good for singular outputs. For classification tasks, it's better to actually inspect the logits.

But my goto setting is always setting min_p at least 0.01 or 0.05! It vastly suppresses incorrect rare random tokens from being created, and it helps massively!

There are no comments on this comment yet.