News for hackers. Stuff that matters.
Re:JSON Schema bundling formalised by csmpltn on Friday September 10, 2021 @06:05AM (#28479553)
We've come full circle back to XML SOAP.

We should consider restricting the registration of .org domains to actual non-profit organizations, and restricting the use of words like "schema" and "standard" to things that have been fully certified as such by internationally accredited engineering bodies.

This doesn't add anything on-top of the tools we've had 40 years ago. We're stuck with this everywhere now, all thanks to pure marketing.

Re:JSON Schema bundling formalised by dgb23 on Friday September 10, 2021 @07:56AM (#28480292)
JSON syntax is more familiar for people who have not been exposed to XML for long.

Parsing and walking a JSON structure is much easier and less ambiguous. It's often very clear how to represent/deserialize/serialize JSON (as-is).

The benefits of using schema are clear, you can add validation and semantics to parse (JSON) into a richer and more consistent data structure.

JSON-schema is in large parts a well designed and clear language that is meant to be read and written by humans. Compare that to DTD, XML schema.

I think in terms of raw functionality, there are very strong parallels. And even though I'm not an XML hater, I find JSON simply more pleasant to work with.

The only fundamental advantage that I would give XML over JSON is the fact that you can encode meta data in XML in a more idiomatic, standard way via attributes. But we all know that is not how XML is being used. If the ecosystem around XML wasn't abusing attributes to encode core/primary data, then there would be a stronger argument for the language IMO.

Re:JSON Schema bundling formalised by da_chicken on Friday September 10, 2021 @09:03AM (#28480924)
> The only fundamental advantage that I would give XML over JSON is the fact that you can encode meta data in XML in a more idiomatic, standard way via attributes. But we all know that is not how XML is being used. If the ecosystem around XML wasn't abusing attributes to encode core/primary data, then there would be a stronger argument for the language IMO.

Attributes are absolutely one of the reasons for XML's failure. They seem so innocuous, until you encounter a schema where the developer created:

  <Entity ID="12345" Type="Person">
    <Property Name="FirstName" Value="Harry" Type="String" />
    <Property Name="LastName" Value="Potter" Type="String" />
    <Property Name="DateOfBirth" Value="1/1/1980" Type="DateTime" />
  </Entity>
And you just want to hunt those people down and slap them.

Nevermind if you've ever encountered XML with mixed namespaces. Then again, reading the article the bit at the end is exactly the same sort of nastiness.

Re:JSON Schema bundling formalised by dgb23 on Friday September 10, 2021 @09:22AM (#28481158)
That's a very illustrative example. It completely and utterly destroys the benefit of having attributes and makes the structure less clear in context of the XML ecosystem.

I imagine if there was discipline around attributes being only used for meta-data and maybe explicit (de-normalized) references (which is arguably meta data, happy to discuss), then XML would be semantically richer by default. However like this we are left guessing the semantics if we encounter some XML.

Somewhat in contrast, HTML is generally, but not consistently, better at making this distinction, which enables clearer semantics around accessibility, HATEOAS and so on. Ironically we are kicking that value proposition into the can when driving web application rendering with JSON (like popular SPA frameworks do), so we have to re-invent meta data and communicate that with JSON-schema.

Re:JSON Schema bundling formalised by oaiey on Friday September 10, 2021 @08:16AM (#28480452)
XSD is not as bad as everyone claims. JSON Schema vs. XSD complexity is IMHO pretty much the same. Working with both over years.

DTD however, let us not talk about that. Let us just forget it.

I think the drama in XML usage was basically that a simple RPC (or resource query) had at least a dozen namespaces, standards and very long element names. But that is not the fault of XML but of SOAP/WS-*. XML itself can be very beautiful and simple.

Oh, and horrible DOM Apis ;). When working in semi-modern .NET parsing XML/JSON is super easy. Doing the same in JavaScript (out-of-the-box) is a drama.

Re:JSON Schema bundling formalised by meltedcapacitor on Friday September 10, 2021 @08:26AM (#28480547)
Namespaces killed xml. Such a bad design. It may have worked out better with every schema required to explicitly flatten in whatever they need from sub-grammars.

I see this json schema thing also seems to use long intricate URLs that you can only copy-paste to identify things... at least in this day and age if one does ID with unwieldy blobs, it'd be good to at least use some hash to get the benefits of content addressing.

Re:JSON Schema bundling formalised by staticassertion on Friday September 10, 2021 @08:36AM (#28480638)
Maybe someone can chime in but parsing XML seems very much more difficult than parsing JSON.
Re:JSON Schema bundling formalised by dgb23 on Friday September 10, 2021 @09:32AM (#28481272)
Just the raw parsing is a solved problem from a user's perspective, the issue arises when you ask yourself "into what" you parse or rather what the semantics of the tree is.

It even might not matter whether you parse it programmatically or not in some cases. Typically when looking at a given XML (or other data structure text representations) you can pattern match the semantics of what you see, but that is not always true, because everyone uses it in a different way, relying almost completely on schema and documentation to convey semantics, which in turn makes the format strictly inferior to JSON, which also doesn't have rich semantics but is simpler.

Re:JSON Schema bundling formalised by oaiey on Friday September 10, 2021 @08:54AM (#28480819)
Parsing XML is just grammar like JSON is ... that is not more or less difficult. JavaScript just has a built-in JSON Parser (which translate JSON into JavaScript object).

As a consequence, JSON in JavaScript is super easy. Using XML is medium-hard.

In .NET using XML or JSON is easy (not super easy).

In C++ using XML or JSON is medium-hard.

Do not know the state of the art for Go, Rust, Java, ...

As a consequence, JSON is - from a JavaScript developer perspective - the holy grail. From a .NET/Java/C++ developer perspective it is just another format.

Re:JSON Schema bundling formalised by radomir_cernoch on Friday September 10, 2021 @09:43AM (#28481432)
In can't agree more.

In Java, there's the JAXB ”parser“ which translates an XML schema into (mostly) straightforward classes. Correct initial setup is the only hard part.

Re:JSON Schema bundling formalised by diroussel on Friday September 10, 2021 @11:20AM (#28482798)
The key difference is that there are no options when you parse json. You just parse it and get some lists and maps back.

In XML you have to decide if you want DTD or schema enabled (which could provide default values, and could be a security risk) and where abs how you resolve the DTDs from etc. Do you resolve references? And many option options.

So JSON benefits not just from a simpler structure, but also from less optionality.

Re:JSON Schema bundling formalised by oaiey on Friday September 10, 2021 @02:05PM (#28485116)
You do not pull schemas dynamically. Schemalocation beyond editors is a pure theory thing. It is not default, it cannot be fast. Any serious developer has the schemas as files in their code when checking the data for compliance.

Basically, in a static typed language you decide for schema checking or not - like for Json -, serializing in structures or access in an API - like for Json.

It is just that JavaScript maps json directly in their dynamic object, array and native types. For xml that is an mismatch. For static languages both are an mismatch.

Re:JSON Schema bundling formalised by staticassertion on Friday September 10, 2021 @04:02PM (#28486291)
> Parsing XML is just grammar like JSON is ... that is not more or less difficult.

My understanding is that that's not the case. XML parsers are Pull or SAX based and it's weird as fuck.

You can abstract that out, but parser complexity isn't something to sneeze out. For one thing, parsing XML is notoriously dangerous - tons of vulns both at a design and memory management level.

Re:JSON Schema bundling formalised by dolmen on Friday September 10, 2021 @08:43AM (#28480705)
> I think the drama in XML usage ...

What about:

- verbosity

- complexity of writing a parser

- complexity of dumping data

- DTDs

- data model which doesn't map directly to a scalar/array/associative array

... and so many other XML features that are either useless or painful just for RPC.

Re:JSON Schema bundling formalised by oaiey on Friday September 10, 2021 @09:03AM (#28480922)
Verbosity is not a thing. We read HTML all the time.

Parser writing and dumping of data is equally complex for both JSON and XML. The only difference is that in JavaScript JSON is so much simpler due to the dynamic nature of the language and the core library. Out-of-the-box XML parsing in JS is just stone-age. Compare that to .NET System.Xml.XDocument and Newtonsoft.Json, then you can see the simplicity how an API could look (in a static type system ;))

No one needs DTDs. No modern contract is every spec'ed in DTDs. XSDSchema has superseeded that 20 years ago. The only issue with it, is that some entry level tutorials are still mentioning it.

The data model mapping depends a lot on the target language. In JavaScript XML is not a match but JSON is. Working with JSON in C++ is equally stupid as working with XML is.

The only stupid feature in XML is namespaces. Without that, it is simple.

Re:JSON Schema bundling formalised by theamk on Friday September 10, 2021 @10:29AM (#28482051)
> Parser writing and dumping of data is equally complex for both JSON and XML.

Strongly disagree. Sure, you can make XML which is easy to parse, but parsing arbitrary XML is hard. Namespaces (and their short prefixes), and custom entities are something that fly in my face any time I use XML but simply do not exist in JSON.

Dumping is usually OK but again, I've seen XML libraries replace short namespace prefixes with full URLs when dumping, making simple documents unreadable.

Re:JSON Schema bundling formalised by easrng on Friday September 10, 2021 @08:25AM (#28480537)
It's easy in JS too, just use a DOMParser
Re:JSON Schema bundling formalised by no_wizard on Friday September 10, 2021 @09:27AM (#28481214)
That is not a JavaScript API, that’s a DOM only API, meaning it’s not reproduced in the language spec. This is important because it won’t work in Node, for example, without a 3rd party library or someone implementing it in Node core.

JSON however is supported in the language spec

Re:JSON Schema bundling formalised by oaiey on Friday September 10, 2021 @02:09PM (#28485152)
And DOM API is an example of bad API design. Compare that to a modern XML API (like .NET System.Xml.XDocument) and you never want to see Dom again
Re:JSON Schema bundling formalised by paulddraper on Friday September 10, 2021 @07:31AM (#28480061)
Do you mean XSD?

XSD = JSONSchema

WSDL = OpenAPI

SOAP = there really isn't a direct analog maybe ""REST""

---

In any case, try creating a SOAP service and WSDL. It's literally so complex that most tutorials have to rely on an IDE to do it. Humans are simply not equal to the task.

Yes, the purpose is similar, but only those who have never used them would think they're the same.

Re:JSON Schema bundling formalised by k__ on Friday September 10, 2021 @07:41AM (#28480142)
I had to use SOAP once... for one API endpoint.

I'd say it was the worst dev experience of my whole career, and I fought with objective-c and app stores.

Re:JSON Schema bundling formalised by m_st on Friday September 10, 2021 @07:53AM (#28480257)
Using SOAP APIs daily since over 15 years with .NET Framework development. Simple and easy to write and then easy to consume through Visual Studio generated code.

Tried some ObjC development early on the iPhone. Gave up quickly.

I agree that SOAP looks like a mess. But given the right tools, it honestly just works. Never had a single issue with these APIs apart being stuck in .NET Framework land.

Re:JSON Schema bundling formalised by kgwxd on Friday September 10, 2021 @08:19AM (#28480474)
Been using it daily myself for just as long with VS. Being basically exclusive to .Net is a huge set back. I would never make a new SOAP services these days. There are other solutions that just work with far less complexity, waste, and limitations.
Re:JSON Schema bundling formalised by paulddraper on Friday September 10, 2021 @08:17AM (#28480460)
And that's the issue.

SOAP isn't some nice standardized language-agnostic API layer.

SOAP is .NET/Visual Studio RPC. (Not technically, but in practice, due to its complexity.)

And hey, great for you. But a lot of people want APIs to be broadly compatible.

Re:JSON Schema bundling formalised by xienze on Friday September 10, 2021 @09:41AM (#28481410)
> SOAP isn't some nice standardized language-agnostic API layer.

What? Of course it is (well, it's a messaging protocol). Where on earth did you get the idea that it's some Microsoft-only proprietary technology?

Re:JSON Schema bundling formalised by theamk on Friday September 10, 2021 @10:35AM (#28482143)
You missed the word "nice" in the GP's message. Also "(Not technically, but in practice, due to its complexity.)"

No one says it is proprietary technology and yes, it is nominally standardized. It is just so complex that it is impossible to use without heavy tooling support, and this tooling is only really usable in Microsoft's technology.

So it seems that people who use MS ecosystem like SOAP ... while almost everyone else who tried it calls is over-complicated, incomprehensible and "worst dev experience of my whole career"

Re:JSON Schema bundling formalised by xienze on Friday September 10, 2021 @11:06AM (#28482608)
> It is just so complex that it is impossible to use without heavy tooling support, and this tooling is only really usable in Microsoft's technology.

Again, gotta call BS here. My early career was 100% SOAP and related WS-* technologies. And it was all in Java, Apache Axis and friends. Wildly popular at the time. Give it your WSDL, out pops a nice Java client. People didn’t like it not because it was difficult to use outside of the Microsoft ecosystem, they didn’t like it because it was the concepts were difficult to understand and there were many complicated technologies piggybacking off of it. It was hard to debug issues, Visual Studio or not. If you had to do something that a generated client could do, it was difficult, Visual Studio or not.

Re:JSON Schema bundling formalised by spookthesunset on Friday September 10, 2021 @12:12PM (#28483606)
Try calling a soap service with python. Oh boy enjoy the pain!
Re:JSON Schema bundling formalised by arethuza on Friday September 10, 2021 @08:59AM (#28480861)
"it honestly just works"

Which I would agree is most of the time - but then it doesn't and you have to try and work out what is going wrong from reams of impenetrable generated WSDL. Which is fun.

Re:JSON Schema bundling formalised by k__ on Friday September 10, 2021 @08:19AM (#28480482)
Yes, I had to use it in a Node.js project.

Would imagine, with the right tooling and TypeScript it wouldn't have been that bad.

Re:JSON Schema bundling formalised by dolmen on Friday September 10, 2021 @08:47AM (#28480751)
Looks like a case of Stockholm syndrom.
Re:JSON Schema bundling formalised by kdtsh on Saturday September 11, 2021 @02:22AM (#28489811)
Over the past year I’ve built an ESB for a client’s deployment of a brand spanking new distribution management system … which you could integrate with only and solely using SOAP. The tooling in Java makes it bearable, but I am still shocked that this system deployed in 2021 has this.
Re:JSON Schema bundling formalised by miohtama on Friday September 10, 2021 @07:52AM (#28480250)
I did XSD validator back in 2003 and indeed this blog post gives me familiar vibes.

I do believe there has to be some sort of common schema format, but I have no idea how this problem could be solved cleanly in practice, even after 20 years.

Re:JSON Schema bundling formalised by sebazzz on Friday September 10, 2021 @08:15AM (#28480443)
And the tools have trouble too. For instance: Java axis2 generating non-deterministic class names (Product1 in one compilation, Product0 in another compilation after).
Re:JSON Schema bundling formalised by ivolimmen on Friday September 10, 2021 @08:06AM (#28480361)
Well let's turn the statement around then: can you create a OpenAPI YAML by hand then? At least with XML and XSD you have proper auto complete.

I really no need see the need to use a schema for JSON. The only thing it has going for it compared to SOAP/XML is the fact the it faster parsing and that it has no schema.

Re:JSON Schema bundling formalised by paulddraper on Friday September 10, 2021 @08:11AM (#28480404)
I can certainly create a JSONSchema (the XSD equivalent) by from memory (except for the $schema URI):

    {
      "type": "object",
      "properties": {
        "foo": { "type": "string" }
      }
    }
And at least in VSCode, I get autocomplete too.

OpenAPI...I personally don't use this as much but yes there are hundred tutorials on creating definitions by hand.

Re:JSON Schema bundling formalised by pionar on Friday September 10, 2021 @09:22AM (#28481169)
Yes, you can. Since OpenAPI itself has a JSON schema, you can get autocomplete from any editor that supports JSON schema.

At my company, we found that the auto-OAS generation tools were too brittle and didn't communicate enough information, so we actually do write our own OAS and run automation to make sure it's accurate.

That's also how our architects communicate with developers about an endpoint - create an OAS for it.

Re:JSON Schema bundling formalised by stueynz on Saturday September 11, 2021 @07:52AM (#28491186)
> At my company, we found that the auto-OAS generation tools were too brittle and didn't communicate enough information, so we actually do write our own OAS and run automation to make sure it's accurate.

I found all OAS spec authoring tooling to be useless toys when building non-trivial real world specs.

The depths of depravity I’m forced to is as follows:

1. Data model in SparxEA because it’s the tool we have in the depths of govt. Generate XSD from tool

2. XSLT to turn the XSD into matching JSON Schema because I’m old and can still remember how.

3. Hand write the OAS with $ref into JSON schema as appropriate

4. Curate sample payloads to be educationally useful

5. Generate CURL and various code samples

6. Bundle it all into approx 80k lines of machine written wondrousness -

7. ReDoc and Mkdocs to build static website with more documentation

All the work is at data modeling, OAS sample curation and mkdocs documentation

Things I wish JSON and Schema had:

1. An equivalent of xs:decimal that JS would use number & double are just plain useless in real world commerce with dollars & cents and bulk item pricing with 3&4 decimal places

2. Has taken the time to fix rfc3399 and give Dates a Timezone suffix. Too many languages Marshall a date into datetime. How many bugs have I seen where it’s broken in the morning and works in the afternoon because the damn server is running in AU with UTC Timezone and we’re in NZ

Re:JSON Schema bundling formalised by dolmen on Friday September 10, 2021 @08:56AM (#28480837)
> can you create a OpenAPI YAML by hand then?

Of course.

> At least with XML and XSD you have proper auto complete.

Tooling for OpenAPI exist. In VS Code for example.

> I really no need see the need to use a schema for JSON

Don't you see the need for precise machine readable description of JSON input and output for a REST API for validation? This is OpenAPI..

Re:JSON Schema bundling formalised by IggleSniggle on Friday September 10, 2021 @08:31AM (#28480589)
You can get proper autocomplete out of OpenAPI, powered by typescript.
Re:JSON Schema bundling formalised by tylercrompton on Friday September 10, 2021 @07:51AM (#28480237)
> We should consider restricting the registration of .org domains to actual non-profit organizations

Too late for that

> We should consider […] restricting the use of words like "schema" and "standard" to things that have been fully certified as such by internationally accredited engineering bodies.

Too late for that and unenforceable

Re:JSON Schema bundling formalised by hyperpallium2 on Friday September 10, 2021 @06:15AM (#28479605)
back to CORBA
Re:JSON Schema bundling formalised by DonHopkins on Friday September 10, 2021 @09:06AM (#28480960)
The World's Second Fully Modular Software Disaster!

It's just riddled with features.

Re:JSON Schema bundling formalised by dnautics on Friday September 10, 2021 @12:59PM (#28484303)
what was the first fully modular software disaster?
Re:JSON Schema bundling formalised by DonHopkins on Friday September 10, 2021 @06:35PM (#28487659)
Glad you asked! ;)

https://donhopkins.medium.com/the-x-windows-disaster-128d398...

[...] X-Windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X-Windows is to memory as Ronald Reagan was to money. Years of “Voodoo Ergonomics” have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards. [...]

X: The First Fully Modular Software Disaster

>X-Windows started out as one man’s project in an office on the fifth floor of MIT’s Laboratory for Computer Science. A wizardly hacker, who was familiar with W, a window system written at Stanford University as part of the V project, decided to write a distributed graphical display server. The idea was to allow a program, called a client, to run on one computer and allow it to display on another computer that was running a special program called a window server. The two computers might be VAXes or Suns, or one of each, as long as the computers were networked together and each implemented the X protocol.

[...]

X-Windows: …A mistake carried out to perfection. X-Windows: …Dissatisfaction guaranteed. X-Windows: …Don’t get frustrated without it. X-Windows: …Even your dog won’t like it. X-Windows: …Flaky and built to stay that way. X-Windows: …Complex non-solutions to simple non-problems. X-Windows: …Flawed beyond belief. X-Windows: …Form follows malfunction. X-Windows: …Garbage at your fingertips. X-Windows: …Ignorance is our most important resource. X-Windows: …It could be worse, but it’ll take time. X-Windows: …It could happen to you. X-Windows: …Japan’s secret weapon. X-Windows: …Let it get in your way. X-Windows: …Live the nightmare. X-Windows: …More than enough rope. X-Windows: …Never had it, never will. X-Windows: …No hardware is safe. X-Windows: …Power tools for power fools. X-Windows: …Putting new limits on productivity. X-Windows: …Simplicity made complex. X-Windows: …The cutting edge of obsolescence. X-Windows: …The art of incompetence. X-Windows: …The defacto substandard. X-Windows: …The first fully modular software disaster. X-Windows: …The joke that kills. X-Windows: …The problem for your problem. X-Windows: …There’s got to be a better way. X-Windows: …Warn your friends about it. X-Windows: …You’d better sit down. X-Windows: …You’ll envy the dead.

Re:JSON Schema bundling formalised by johnnycerberus on Friday September 10, 2021 @06:56AM (#28479826)
> CORBA

"Well, it's high noon somewhere in the world."

Re:JSON Schema bundling formalised by ChrisMarshallNY on Friday September 10, 2021 @06:31AM (#28479675)
> CORBA

Get off my lawn!