I'm not a native speaker, but as far as I know a "boffin" is not derogatory, at least in British English. I liken it to "nerd" in US English.

From Wikipedia[^0]:

> Boffin is a British slang term for a scientist, engineer, or other person engaged in technical or scientific research and development. A "boffin" was viewed by some in the regular services as odd, quirky or peculiar, though quite bright and essential to helping in the war effort through having and developing the key ideas leading to transformative military capabilities.

[^0]: https://en.wikipedia.org/wiki/Boffin

You may be confusing boffin (British slang for expert) with buffoon (a ridiculous but amusing person)

Thanks you are correct, I took it as derogatory.