News for hackers. Stuff that matters.
Re:My 2.5 year old laptop can write Space Invaders in JavaSc... by israrkhan on Tuesday July 29, 2025 @07:13PM (#44729336)
do you have a suggestion or a way to measure if model capabilities are getting destroyed? how do one measure it objectively?
Re:My 2.5 year old laptop can write Space Invaders in JavaSc... by RALaBarge on Tuesday July 29, 2025 @08:05PM (#44729673)
Ask it a series of the same questions after you train that you posed before training started. Is the quality lower?
Re:My 2.5 year old laptop can write Space Invaders in JavaSc... by israrkhan on Wednesday July 30, 2025 @02:38AM (#44731429)
That series of questions will measure only a particular area. I am concerned about destorying model capabilities in some other area that that I do not pay attention to, and have no way of knowing.
Re:My 2.5 year old laptop can write Space Invaders in JavaSc... by simonh on Wednesday July 30, 2025 @03:26AM (#44731676)
Isn’t that a general problem with LLMs? The only way to know how good it is at something is to test it.
Re:My 2.5 year old laptop can write Space Invaders in JavaSc... by mensetmanusman on Wednesday July 30, 2025 @08:14AM (#44733183)
These are now questions at the cutting edge of academic research. It might be computationally unknowable until checked.