Re:Parsing JSON Is a Minefield (2018)
by blackflame7000 on Saturday August 17, 2019 @05:44PM
(#20726609)
There is no functional difference between an array of unsigned bytes (binary) and a array of signed bytes (char data). The only difference is that when you send binary, 0 is now a valid value instead of null terminating a string. Therefore you must prepend the size because you can no longer parse till you find a NULL byte. It is always safer to know the size you must allocate ahead of time instead of dynamically growing a buffer until the text stream is terminated.
[ Parent
| on HN ]
There are plenty of examples of binary formats where you do not know buffer sizes until you've received all the data, and where assumptions with parsing the data can cause a buffer overflow.
decompression and PNG libraries for example have caused massive security impact across the industry because of reuse in different products. Font handling, compressed bitmap, and windows cursor parsing also have been sources of issues.
Mozilla didn't just invest in Rust because parsing HTML and JSON are hard. Its all hard.
Re:Parsing JSON Is a Minefield (2018)
by blackflame7000 on Sunday August 18, 2019 @05:04AM
(#20729106)
“It is always safer to know the size you must allocate ahead of time instead of dynamically growing a buffer until the text stream is terminated.”
And then you go on to give examples of said scenarios of how this is true while saying I’m wrong? Anytime you have an unknown payload you have to make a determination of how long you’re going to wait, how much you’re going to accept, buffer, etc before it’s become a drain on the system
You can still reserve one value as a control character and have non-text streams that don't need prefixing of data frames by size.
These are also easier to recover the data from in case of corruption, or re-sync the receiver, since you have a control byte you can sync on.
Re:Parsing JSON Is a Minefield (2018)
by blackflame7000 on Saturday August 17, 2019 @11:52PM
(#20728134)
Yes, in fact, this is most common in video streaming formats. These types of streams are more commonly downloaded as opposed to uploaded where the server has to be careful not to exhaust too many resources parsing variable-length messages.
It's also fairly common if you have alway-on streams without any signalling, where you can connect to the stream at any time, like with UART.
Re:Parsing JSON Is a Minefield (2018)
by paulryanrogers on Sunday August 18, 2019 @12:00AM
(#20728159)
JSON over HTTP can communicate its size with an HTTP header.
Re:Parsing JSON Is a Minefield (2018)
by blackflame7000 on Sunday August 18, 2019 @05:08AM
(#20729114)
There’s no reason strings cant be sent like binary if you do a size header first. The problem is trying to send binary like a string where your data might have 0s that could be interpreted by the receiver as string terminations. Typically base64 is used to address this issue
Re:Parsing JSON Is a Minefield (2018)
by edflsafoiewq on Sunday August 18, 2019 @09:44AM
(#20730089)
Even if it's compressed?