wwaites
Re:CKAN - The CPAN of data
on Wednesday April 06, 2011 @10:08AM (#2415105)
seems to be back up now... the http headers suggest caching with a time of >30 mins is now in effect... i think caching and cache poisoning is generally hard to get right for dynamic data-driven sites...
Re:Aggregating Data Registries with Go
on Wednesday March 30, 2011 @10:37AM (#2387393)
Thinking about this some more, I think to replace model.Package with an interface. The existing struct of that name will become JsonPackage that implements it. This is because the CKAN representation of a dataset has some modelling problems (that are being addressed in parallel by the CKAN team) and it isn't clear that the existing models as represented in the JSON should overly inform the functioning of the ckand aggregator.
Re:Aggregating Data Registries with Go
on Wednesday March 30, 2011 @10:33AM (#2387365)
I think it depends. For the class of "quick hack to get some data from a JSON API and munge it" having to model the data can be a burden. Just using a Python dict and getting the values out is initially quite a lot simpler.
However for anything more complicated than that you start having to model the data anyways, in Python or Go or any other language otherwise it will quickly devolve into unmaintainable spaghetti. In this case, I was forced to do this early on, which made me think through the structure of the program more thoroughly than I might otherwise have done.