It's also non-extensible so these types it's missing cannot be added without assumptions or overhead from convention.
Transit is a good solution to this.
1. ISO UTC 2. ISO local 3. Epoch 4. 2018-04-20 5. Jan 01 2016 11:58am
Probably others that I can't recall. It's a damn mess.
I never understood why simple integer unix timestamps are not more prevalent in API-s? Or some monotonic count from any reasonable epoch, depending on the context. How many APIs really have to ever return dates predating unix?
That will probably cause a twitch to any dev that used Asp.NET up until a few years ago.
A timestamp is a point in time, whatever the time is in Paris or Tokyo. It is an abstract value and it is way better this way.
A timezone is a filter through which you display your timestamp, and it tells you what your local time was when that timestamp occurred.
So yeah, always store and exchange time information as timestamps. Timezone is extra styling information, just like CSS.
There are lots of examples like this. Some time events you want to happen at absolute time points (in which case time zones are only for display purposes), but very often your events need to be ”time-zone aware”. There is no universal rule, and thinking that there is will lead to all sorts of trouble.
A timestamp is a point in time, and is the same everywhere (excluding relativistic effects).
What you describe is definitely a valid use case (and having developed software that programs hardware that is controlled by a calendar, I have painful experience in this). However, it's not a timestamp. I'd call it time-of-day or wallclock. Some systems like SQL refers to it as simply "date" and "time". Whatever you prefer to call it, it's not a timestamp.
How you represent it doesn't change the definition. It could be stored as the number of seconds since some arbitrary point in time, such as Unix timestamp, or a Julian timestamp.
Or, it could be a given time and date combination with some fixed point of reference, such as UTC. I'm sure we all have favourite ways of representing timestamps.
When I set an alarm for any time in CEST in 2022 and convert this to UTC before saving it, it will very likely ring at the wrong time, simply because CEST will probably not exist by then and be replaced by CET due to the EU getting rid of DST.
You might not have encountered such scenarios but they are very much out there.
Expressing time based on the standard reference timezones (i.e., UTC) is not stripping the time zone away.
Example: The EU will most likely get rid of DST around 2022. Any time you set beforehand in CEST or CET will be an hour off, depending on which the EU will get rid of.aybe the EU doesn't get rid of DST and keeps it, you don't know. So unless you have a time machine, you cannot convert the time to UTC.
> Timezone is extra styling information, just like CSS.
This is an incredibly reductive view of time data and the possible applications that use it.
Not all time data is timestamps, and UTC timestamps strip out information that is not "styling".
Dates should not be encoded with time zone info anyway, as timezones is context-dependent. Dates should be encoded in UTC and then let clients interpret them according to their context.
You alarm will be an hour off when you convert to UTC beforehand.
I dunno, how do you distinguish "April" (the month) vs "April" (someone's first name) or 18 (kg) vs 18 ($) vs 18 (-th of the month) vs 18 (page number)?
> It is lower fidelity (seconds vs milliseconds).
This isn't a problem if you have a conforming implementation (2^52 milliseconds is over 100'000 years), but as nitrogen pointed out, you do apparently have to worry about that.
This is a frequent gotcha with Typescript, where even if your type is declared with a Date field, Javascript won't care when it deserializes it, as that type information is all erased and not available at runtime.
(Also, Number in JS being floating point and all, it lacks the integer precision for high resolution timestamps - if you start serialising 64 bit timestamps and expect a JS-style runtime to do good things with them it doesn't end well.)
Ah, thanks, that makes sense; I'd forgotten that Javascript had a built-in Date type. (Strictly speaking, then, you ought to be able to write something like `{"time":new Date(...)}`, but that obviously doesn't work in practice.)
Types! Extensible types. Like I said above, Transit is a good solution to this. With a fixed set of types, arbitrary data will always have ambiguity.
Current precision with a 32-bit float (JSON/JavaScript are 64-bit usually, though in non-JS it's common to use Bigdecimal in slight non-compliance) is much worse than one second.
That's the point, timestamps should be self-describing. If "look up the structure in documentation" suffices we might as well just use protobufs.
Maybe OP meant timestamp, but that's not what they wrote.
Read the spec. Basically all it says is that the numeric type is a float, but doesn't say anything about its precision.
I'm amazed that there haven't been any security vulnerabilities found yet that takes advantage of different number formats in different JSON implementations.
You'd think that a data serialisation format should get at least numbers right. All the ones that came before (and after) did, yet we're still stuck with JSON.
As it turns out, the only way you can reliably store an integer in JSON is to use a string field. This is incidentally also the correct way to store a username in JSON.
The problem with JSON is that it's just human-readable enough that people think it's a config file format. It's not.
That attitude is misguided. Many people do in fact use JSON as a config file format, so it is a config file format. Real world usage outweighs prescriptions about what is and isn’t correct.
Looking at the first standardized JSON spec, it’s not particularly prescriptive about usage, simply saying:
JSON is syntax of braces, brackets, colons, and commas that is useful in many contexts, profiles, and applications.
Of course, it would be a better config file format if it supported comments, but people use it even despite that. I think we should try to understand the reasons for that rather than just telling people they’re doing it wrong.
JSON is one of the worst config formats, even worse than the Apache or nginx config formats. Not being able to simply comment out a config line temporarily for testing purposes or to leave a explanatory comment for special setting is simply bad.
Still, everyone forgets HOCON [1] is a thing, which solves many of the problems expressed here about JSON -for configuration-. It's easy to clearly specify things like time, reference other parts of the config, or if you want to change just one value, you can add that to the 'end' of the HOCON file and be GTG.
[1] - https://github.com/lightbend/config/blob/master/HOCON.md
Skimming through it, concatenation of unquoted values is where it goes off the rails for me. This is quite the gotcha: https://github.com/lightbend/config/blob/master/HOCON.md#not...
My wishlist for a friendlier JSON would be, in priority order:
- Allow trailing commas
- Comments
- Allow newlines instead of commas
- Allow unquoted dictionary keys
And I think that’s it. I’m not even sure the last one is worth the extra complexity.
I don't know why people would prefer that over YAML or others but at least you can actually add comments in them and the reason for their popularity doesn't seem to be baked in parser support because these tools are adding their additional "JSON" parsing anyway.
...and those people need to face the consequences of their poor and missguided technical decisions.
In this day and age we have no excuse to repeat the "but everyone is using XML for that" mistake. Just pick the right tool for the job and stop complainig that the tool needs to change to compensate for your poor judgement.
If you want comments, pipe it through json5 first or something. VSCode again supports comments as a courtesy. More tools that use JSON are starting to as well.
It's just not a big deal.
That's just tooling compensating for the shortcomings of a format being shoehorned into a use-case that falls outside of it's scope.
It's the XML nonsense all over again.
> It's better done and more mainstream than any other solution I've seen compared to people saying "well technically you could build that for <pet format>."
That's the same short-sighted line of argument that was used to force the mistake of using XML everywhere.
There are right tools for the right job. JSON os the right tool for a lot of jobs, but config files is not one of them.
Even though it’s not ideal, I actively prefer JSON over every other random format I’ve come across. This is speaking as somebody who mostly has to tweak existing configs, rather than extending them or writing new ones.
YAML (along with I guess TOML etc) looks nice at first glance, but it has too many weird syntax shortcuts that make it hard to figure out what’s actually going on. And googling for syntax like “[ ]” is hard!
With JSON, the data model is super simple, and for a given set of data there’s only one way to write it down. Sometimes the data model is too simple, sure. But even for fiddly cases like dates and times, there’s often an obvious solution (in this case, ISO 8601 strings).
You have to look at JSON in a historical context to understand it. It exists so that someone could get some data into their Javascript program with "eval". It was then standardized so that you didn't need a full Javascript engine to understand it, because Python and Perl and Ruby didn't have one, and it turned out that evalling random data from the Internet was a security disaster. Does that make it a good human-readable config file? Nope. Does that make it a good interface definition language? Nope.
It exists because it got popular early, and now we're stuck with it. Now it's too late to apply band-aids to make it a human-readable config language or a good computer-to-computer interchange format, because you will always have old parsers around and people will naturally want to target those. Trust me, 99% of developers will moan when you tell them that they have to use GRPC+Protos to access your API. They will be equally mad about your JSON extension that has comments and integers in it, because the dialect of Brainfuck they use for all their projects doesn't have a library that supports those extensions and their editor can't syntax-highlight it or autoformat it.
I would like to give you a solution to this problem, but there isn't one. JSON "won", but it's bad at everything except being well-understood. That seems to be all that people really care about.
I call BS.
As a counterexample search for the word "comment" in https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html. Or are you saying that http is not a computer-to-computer interchange format?
It turns out these "comments" were a disaster and user agents are moving away from even providing them. You put "firefox" in the user-agent header, and web servers would read the comment and send you "your browser doesn't work" instead of the actual document. That did not make for a very compatible web... turns out comments in machine-to-machine communication are a bad idea.
How about ".comment" sections in ELF binaries, do they count?
But that doesn't change the fact that people put comments into machine protocols.
For another example, lots of machine to machine protocols specify XML. Such as SOAP. They therefore all support comments.
In accord to the general trend, eventually they get abused into having a semantic meaning. For example https://support.ptc.com/help/windchill/wc111_hc/whc_en/index... shows how one system uses comments in SOAP to create documentation and a WSDL that lets interfaces to your code to be automatically generated.
And so comments eventually become executable. But that doesn't change the fact that comments can exist in machine to machine formats.
Do we use the same Json me and you?
> Json is an open standard file format, and data interchange format, that uses human-readable text to store and transmit data objects
- https://en.m.wikipedia.org/wiki/JSON
Json is text based, and meant for storage and transmission in a human readable format.
> Although Douglas Crockford originally asserted that JSON is a strict subset of JavaScript, his specification actually allows valid JSON documents that are not valid JavaScript; JSON allows the Unicode line terminators U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR to appear unescaped in quoted strings. > JSON is a strict subset of ECMAScript as of the language's 2019 revision.
https://en.m.wikipedia.org/wiki/JSON#Data_portability_issues
But crockfird said it was done to keep the parsing simple (and thus secure), and between seeing how XML has it, and seeing how poorly many json parsing implementations are despite this simplicity, I have begrudgingly decided to accept it. Still wish there was a way without those problems though.
I think this was stupid. First, it happens anyway to some extent with people just using custom byte encodings for "strings". Second, using comments for metadata could be useful -- even if optional. For example, adding types after strings to specify things like the date format.
Not true for most numbers. A 64-bit IEEE float can handle any 53-bit integer with no error at all.
That's webtech in a nutshell isn't it?
Nobody says "what a shame I can't open that jpeg image in a text editor".
Everyone understands that an image is an image, and there are tools for dealing with images, like photoshop.
The same could be said for data interchange formats like json.
Nobody except me, of course. I'd be thrilled if I could open up a JPEG in Emacs and e.g. view/edit the EXIF metadata in a text buffer (and/or pop up a graphics buffer for editing the image itself). Similarly, I'd be thrilled if Emacs was able to parse MessagePack data and let me view/edit it in a buffer. Both of these things are theoretically possible, but to my knowledge nobody's actually done them yet.
Granted, calling Emacs "just" a text editor is pretty stretchy, but still.
JSON vs. Bencode, JSON can get you the data good enough a lot faster, while a bencode parser is easier to write. My feeling is that most formats with schemas usually fail somewhere in the details anyways. Honestly I think I want too much from them; e.g. the YANG Data model language is a kitchen sink but lacks a lot.
We're not. We're gzipping the json when we transmit it.
and complete.
Both are minimal self describing data serialization formats, but JSON is incomplete. It misses a type to represent one of the fundamental type: byte blobs. And also parts of floats.
Which means there are a lot inconsistent ways to hack that in e.g. base64 encoded strings. But this means it's loosing partially it's property of being self-describing (same if you allow numbers as strings e.g. "-12").
Just to be clear I'm not saying JSON should have bytes directly in it. But a "native" base64 string additional to string, number, null, list and map would help. E.g. `{ "bytes": b"YWJjZGU=" }`
Wrt. floats it's missing (+/-) Infinity, NaN is more like a error variant so it's okish to be missing, but then why not have it.
Also for completeness it would be better to differentiate between int and float as float is imprecise due to rounding errors.
----
PS: I don't like `null` but having it in this kind of data serialization format is still required. But the difference between a field not being there and it being `null` can be a mess and not all tools handle it well.
PPS: Both JSON and MsgPack can be used for non self describing serialization, e.g. by serializing the a record (fixed number of fields in known order) as a list of values instead of a mapping. But both are focused on enabling self describing serialization.
EDIT: PPPS: Yes, it's a very weak form of self-describing is use here, there are systems which are much more self describing e.g. XML+XML Schema linked from the XML document but also tend to be far more complex