> I was decoding a large JSON data structure that could
> easily have thousands of different fields nested 10+
> levels deep. Go's standard json library usually
> recommends that you define a struct type to hold the
> result of the decode, tagging field names with the
> `json` annotation. This obviously doesn't work when you
> only care about a few dozen of the several thousand
> possible fields.
Sure it does! type Response struct {
LevelOne map[string]struct{
LevelTwo map[int]struct{
Value string `json:"eventual_value"`
} `json:"level_two"`
} `json:"level_one"`
}
The response can have way more fields than just the ones you declare in your struct. response['level_one']['level_two']['eventual_value']
?I did ignore fields that I wasn't interested in; however, you do at least have to declare all the intermediate structs in the tree. With 10-ish levels of nesting, that's a lot of boilerplate; even in your example with 3 levels, you've still got 6 lines of code for what's a one-liner in Python.
It wouldn't be as much of a problem for a production system where you define your data structures once and then expect to amortize them over many, many changes. But I was prototyping, and the point of prototyping is to not spend time on boilerplate that isn't necessary to prove out your concept. A few hundred lines of type declarations just so I could decode JSON isn't exactly convenient.
But, at your insistence, we can keep digging. It's easy to write a polymorphic function that arbitrarily picks out values from a Go `interface{}` value.
func lookup(v interface{}, keys ...string) interface{} {
switch len(keys) {
case 0: return v
default:
dict, ok := v.(map[string]interface{})
if !ok {
panic(fmt.Sprintf("%T is not a dictionary", v))
}
return lookup(dict[keys[0]], keys[1:]...)
}
}
And then we can use it like you would in Python: lookup(response, "level_one", "level_two", "eventual_value")
This doesn't completely remove the burden of type asserting (unfortunate), but it at least makes nested lookups much easier. Arguably, the nested lookups are probably where the type asserts hurt the most.Full example: http://play.golang.org/p/4tWtqGRgU6
Yeah, prototyping with JSON is one place where dynamic languages are more nimble.
For example, the Python bindings to Amazon S3 let you access files through dictionary notation. BeautifulSoup lets you iterate over child nodes with a for loop, or access attributes as a dict. NumPy lets you use standard arithmetic operators on matrices. Judicious use of this in libraries makes the resulting code much briefer.
Yes, this is another specific area where dynamic languages do prototyping better and can produce shorter code. I did an exercise where someone implemented a routine in Clojure that I had written in Python. The Python was shorter! In part, this was due to list comprehensions. In part, it was due to an excellent library. What you describe is what many Smalltalkers did as well: create your own control structures and DSLs.
I've sometimes wondered what could be done with a dynamic superset language of Go, such that one could mostly add type definitions and compile to Go.
It should be simple enough to implement a dynamic JSON parsing DSL in Go. It would be slower and memory inefficient, but much more convenient for rapid development.
Python just hands you back a dict of lists, and you pull out the information you want, no type declarations needed.
The "user-defined types like built-ins" lets you build libraries where you can use the basic language structures on the data structure itself, without having to convert to an intermediate form. For example, the Python bindings to Amazon S3 let you access files through dictionary notation. BeautifulSoup lets you iterate over child nodes with a for loop, or access attributes as a dict. NumPy lets you use standard arithmetic operators on matrices. Judicious use of this in libraries makes the resulting code much briefer.