How is it possible that they were getting malformed input? This was happening in go-to-go communication, or was there some kind of cross-language interop?
Any idea why the performance was so much slower than JSON in your case? The technique described in the OP would seem to make that impossible.
Do you think it's possible the database column type or collation was somehow affecting the gob?
The docs explicitly mention that invalid input will make it panic and that can be confirmed by reading the code or fuzzing the input.
From my understanding, there is no compile time schema so everything is done with runtime reflection and that is bound to not be super fast. Granted, JSON is the same on paper, I would guess that the JSON package had more eyes on it and optimizations.
In our case, everything was using JSON except this one component due to some historical oddity so it was also a win in terms of simplifying.
It panics on malformed input which is a no go for us since high availability is really important for us, and it showed quite a lot in the performance and memory profiles (roughly 5 times the time and memory as doing the same with JSON).
The code was converting some data to gob, and storing it in the database for later.
We now just do the same but in json, it’s human readable and Postgres validates that the data is valid JSON.
And unmarshaling it does not panic.