Re:Non-determinism in GPT-4 is caused by Sparse MoE
by MacsHeadroom on Sunday August 06, 2023 @04:22AM
(#37019995)
I follow a lot of highly respected researchers and (research minded) operators in the fields I'm interested in. Very often they post about papers of interest on X/twitter or their personal blogs. I also follow a handful of science communicators on YouTube who do short summary videos of papers of interest (Two-Minute-Papers, Anton Petrov, Sabine Hossenfelder, to name a few).
Other times I notice a general trend (ex. increasing discussion of a new paradigm X, more startups raising to work on Y, or a large chunk of talks at an annual conference being variations of Z).
Then I ask the forementioned academics and operators in my circle what papers I should read to get a handle on XYZ and/or simply follow the citations.
Given the amount of followers a lot of these researchers, operators, and science communicators have, I do not think I'm remotely unique in my efforts.
Other times I notice a general trend (ex. increasing discussion of a new paradigm X, more startups raising to work on Y, or a large chunk of talks at an annual conference being variations of Z).
Then I ask the forementioned academics and operators in my circle what papers I should read to get a handle on XYZ and/or simply follow the citations.
Given the amount of followers a lot of these researchers, operators, and science communicators have, I do not think I'm remotely unique in my efforts.