Re:Analyzing multi-gigabyte JSON files locally
by Yoric on Saturday March 18, 2023 @02:20PM
(#35211692)
Does Go perform interning of strings?
Re:Analyzing multi-gigabyte JSON files locally
by Groxx on Saturday March 18, 2023 @02:41PM
(#35211871)
* * *
Re:Analyzing multi-gigabyte JSON files locally
by xh-dude on Saturday March 18, 2023 @10:03PM
(#35215252)
Use of Go’s runtime reflection to produce a struct definition would result in one instance of the key string, many instances of the numeric value
This approach is available/implemented in the standard library, the whole story with JSON in Go is includes a few different approaches but this is definitely anticipated.
The map has its two slices (pointer + length + capacity = 8*3 times two slices) and the string needs a separate allocation somewhere because it's essentially a pointer to a slice of bytes. All of which is true for almost all reasonably efficiency-focused languages, Rust and Go included - it's just how you make a compact hashmap.