Re:My 2.5 year old laptop can write Space Invaders in JavaSc...
by israrkhan on Tuesday July 29, 2025 @07:13PM
(#44729336)
do you have a suggestion or a way to measure if model capabilities are getting destroyed? how do one measure it objectively?
[ Parent
| on HN ]
Re:My 2.5 year old laptop can write Space Invaders in JavaSc...
by RALaBarge on Tuesday July 29, 2025 @08:05PM
(#44729673)
Ask it a series of the same questions after you train that you posed before training started. Is the quality lower?
Re:My 2.5 year old laptop can write Space Invaders in JavaSc...
by israrkhan on Wednesday July 30, 2025 @02:38AM
(#44731429)
That series of questions will measure only a particular area. I am concerned about destorying model capabilities in some other area that that I do not pay attention to, and have no way of knowing.
Re:My 2.5 year old laptop can write Space Invaders in JavaSc...
by simonh on Wednesday July 30, 2025 @03:26AM
(#44731676)
Isn’t that a general problem with LLMs? The only way to know how good it is at something is to test it.
Re:My 2.5 year old laptop can write Space Invaders in JavaSc...
by mensetmanusman on Wednesday July 30, 2025 @08:14AM
(#44733183)
These are now questions at the cutting edge of academic research. It might be computationally unknowable until checked.