You'd be hard-pressed to be able to blame ASN.1 with XER (XML Encoding Rules) encoding since then you'd throw an XML parser at the problem, side-stepping the bit twiddling and exchanging it for XML parsing bugs instead. Similarly, if all you have is fixed-length items, OER (Octet Encoding Rules), you essentially get a formally specified struct with a well-defined way to go from there to XML or JSON with JER (JSON Encoding Rules).
I imagine we'll see this happen with CBOR (whose ASN.1 equivalent is technically CDDL) a few years down the road.
Edit: would be really happy to see some good open source compilers with extensible encoding support being listed here
I think that's the reason of ASN.1 being trapped in a very restrictive space. From Olivier Dubuisson's ASN.1 -- Communication between Heterogeneous Systems, p. 86 et seq. talking about use of ASN.1 by the IETF:
> the definition of many macros and macro instances to represent semantic links instead of information object classes and information objects although no ASN.1 compiler properly takes into account the macro concept (on the other hand, no compiler of the public domain does with the information object class concept unfortunately);
In essence, ecosystem failure due to a non-existent overlap of the telecom industry (full of commercial and expensive solutions) and the Internet community (working in a standards committee that tracks pre-existing permissionless innovation distributing the standards at no cost, strongly affiliated with free software ideas) caused ASN.1 to fall by the roadside.
Now that ASN.1 is considered uncool, the ecosystem certainly won't be coming anymore, so we'll be forced to reinvent ASN.1, probably re-learning every lesson along the way.
Yep. I think this is the crux of the matter.
I don't even think it is an "uncool" thing. In a word, IMHO it's too "closed". Xml is not exactly hip, but I can find high scoring python xml libraries on snyk.io. I know that is not a very scientific metric but it's a good proxy for a bunch of things that are hard to measure.
I think even if it were "cool", it would be a struggle to get a culture to nucleate. It has the same vibe as codecs: complex, kinda arcane, too closely associated with closed source, litigious, corporate culture. It doesn't help that it is guilty by association with numerous poorly written libraries. Unless that is what you mean by "uncool".
Now that there is cbor, msgpack, cap'n proto, protobuf, thrift, and a bunch others (just in the binary space), asn.1 would have to make a really compelling argument to win out. It would need a lot more than coolness.
I think this already happened with Google Protocol Buffers
This is a shameless plug, and not a compiler per se, but it's pretty close[0]. It's a codec framework for ASN.1 in Rust similar to `serde` for other formats if you're aware of that. It uses Rust's traits to define seperate layers for the codec model, and the encoding rules, allowing you to have one model used with many encodings, and share one decoder & encoder for all models. Extensibility just uses the regular Rust `#[non_exhaustive]` attribute. It doesn't support formats like A/UPER yet, but that is coming in the next year. :)
I don't know of any third-party projects that are using it, though from talking to users and looking at the download numbers there are quite a few, they're just closed-source. You can look at the dependent downloads[0] to get an idea of whose using the implementations included with rasn.
> How easy is it to use as a codec layer from other programming languages?
It's as easy as using any Rust code from other programming languages. Rasn is completely IO-agnostic, it works entirely using byte slices (`&[u8]`). So all that should be required is writing a C-friendly layer if doing this by hand, or using a language specific binding library like PyO3 to generate more friendly bindings.
While attempting to implement clean auditable code which would accept all signatures openssl accepted and reject all that it rejected years ago I fell down a rabbit hole of trying to discover if there was any complete and correct open source implementations of BER. I audited dozens of them and was not able to find a single one. They all implement different subsets and have various flaws in the more weird/useless ways of encoding things.
This effort eventually result in an openssl CVE, and we never got a consistent implementation: the exact set of messages accepted was far too irregular and dependent on the implementation.
(OpenSSL eventually 'fixed' the issue by restricting the set of accepted inputs ... in an uncoordinated manner guaranteed to create a vulnerability for any system where consistent validation is security critical.)
ASN.1 isn't unique in this though. Almost all other complex parsed formats with multiple implementations have troublesome inconsistencies too. The gap between "make it work" and "make it always work correctly" is too big. People make code that is sufficient for the purposes they care about and then it gets deployed into places where it's not sufficient but good enough to look like it is for a little while.
As the saying goes, "don't shoot the messenger". There's more quantity than quality when it comes to ASN.1 implementations, especially free ones, and that's largely where the impression is coming from. DER in particular is not hard to parse, but people don't seem to bother with thinking carefully about the edge cases and such.
If you were to wave a magic wand and eliminate from the world all messages that are encodings of ASN.1-defined values, disaster would certainly strike on a scale far beyond any that the most pessimistic have described for possible effects of the Y2K computer bugs. Aircraft would collide, mobile phones would cease to work, virtually all telecoms and network switches would be unmanageable and unmaintainable and would gradually die, electric power distribution systems would cease to work, and ... smart-card-based electronic transactions would fail to complete and your washing machine might fail to work ... and your life would become a misery!
--
It's suffice to say that them security people have proven, and unfortunately are likely to keep proving him to have been right on more than one occasion.
[0] https://www.oss.com/asn1/resources/books-whitepapers-pubs/la...
[1] https://www.erlang.org/doc/apps/asn1/asn1.pdf
[2] https://www.erlang.org/doc/apps/asn1/asn1_getting_started.ht...
Or day drinking and quitting your job, depending on the mood.
ASN.1 done right is pretty easy to secure, and much less error prone than pretty much any text format
Analyze code: peer review, bounty, ...
Harden through invariant checks and logic checks.
Make your system/code anti fragile by detecting wrong encoding, reporting it (uh-huh, yeah, think that any code even reporting/logging is an attack surface, hint hint log4j) and treating future traffic from source of attacks as a way to train your defense.
Create a culture of security: simple architecture, design workshops, threat modeling, thinking about what someone with attacker mindset would do, value experience (coming defense source) AND involve newbies (they will try again what didn't work in the past, and sometime it works)
Quoting http://seriot.ch/projects/parsing_json.html:
In conclusion, JSON is not a data format you can rely on blindly. I've demonstrated this by showing that the standard definition is spread out over at least seven different documents (section 1), that the latest and most complete document, RFC-8259, is imprecise and contradictory (section 2), and by crafting test files that out of over 30 parsers, no two parsers parsed the same set of documents the same way (section 4).
Here's an example of an actually simple format where it's easy to define a canonical representation: https://en.wikipedia.org/wiki/Canonical_S-expressions; it was used for SPKI (https://theworld.com/~cme/html/spki.html) which sadly never went anywhere but is much more pleasant than the ASN.1/X.509/JWT etc mess that actually won.
That link is not an issue with json, but with javascript (that js treats keys named __prototype__ special). Its one of the most famous js security bugs, and also applies to xml and whatever else if you dont implement the parser carefully.
JSON is meant to be (and was widely advertised as) a Javascript subset, but isn't, but almost no one is aware of this. So of course it's also an issue with JSON. It's also not the only divergence that leads to security problems, in fact if you follow the first link in the comment you will see that this was posted in the context of trying to fix another misspecification in an attempt to actually make JSON a proper subset (which turned out to be impossible to completely accomplish, although at least the paragraph separator issue has been fixed in 2019). See here:
https://www.trustwave.com/en-us/resources/blogs/spiderlabs-b...
A fun historical one was remote code execution due to a json parser bug in Google Chrome. Oops. If you try a simple search on mitre, you'll see some false positives, but also a bunch of actual vulnerabilities each year:
https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=json
Here is a nice blog post walking through various JSON based exploits:
https://bishopfox.com/blog/json-interoperability-vulnerabili...
I'll just leave this here.
Just because something has an RFC number doesn't make it good.
I get the impression this partially due to a) most implementations being adhoc, of varying completeness, and written in C or C++ (e.g. OpenSSL); and b) sheer complexity of ASN.1 and other standards from that family/era.
Sources: following software security researchers/practioners on Twitter, a few podcasts, here, etc.