News for hackers. Stuff that matters.
Re:Jolt – JSON to JSON transformation by otabdeveloper1 on Wednesday November 23, 2016 @07:14AM (#13022006)
Are we _really_ going to reinvent all the spectacularly bad ideas of XML, except this time in JSON?
Re:Jolt – JSON to JSON transformation by DonHopkins on Wednesday November 23, 2016 @08:11AM (#13022363)
I think you have make a distinction between "bad ideas of XML" and "essential ideas of CS that were implemented in XML because it was the fad at the time, and SGML before that, and are now being reimplemented in JSON, and will be reimplementing in the next big thing".

It's like complaining about regular expressions "Are we _really_ going to reinvent all the spectacularly bad ideas of Perl, except this time in Python?

Re:Jolt – JSON to JSON transformation by falcolas on Wednesday November 23, 2016 @09:12AM (#13022731)
The fact that we keep re-implementing ideas like this means that there are a lot of valid use-cases for this kind of tool. The same applies to (JSON|XML)RPC, xpath|jq, and schemas. There are even folks currently developing namespaces for JSON!

I think it's human nature, as often as I see it repeating throughout history (and not just in the tech industry). Just wait, there will be a new JSON just over the horizon which will close the circle and bring back a very simplistic serialization format with a minimal number of built-in structures.

Perhaps it would be better to create a C++ of serialization formats, all the features out of the gate, so you can pick and choose from the beginning. Again, I think human nature means we can't go back to XML (too much horror associated with SOAP and WSDL), or even continue with JSON (custom and subtly incompatible parsers being built around the core to handle missing features).

Re:Jolt – JSON to JSON transformation by niftich on Wednesday November 23, 2016 @09:40AM (#13022926)
This cycle keeps re-occurring in multiple places every few years because the ideas are in fact useful, but the surrounding ecosystem falls out of favor.

Old solutions accumulate cruft and start to need more expertise (and sometimes, straight wisdom) to operate well. People complain, blog posts are written, inertia and inaction are challenged; babies are thrown out of the bathwater as clean-room rewrites commence with different stacks, different tradeoffs, and often, different bikeshedding.

The new, shiny tool is promoted on its merits; some of that excitement inspires hype, which takes on a life of its own. Through a combination of informed usage and less-informed codemashing, the tool accumulates users from all walks of life, who turn into willing or unwilling stakeholders in its future. Development pressures pull the product in different ways, some people leave, eventually only the really passionate or really locked-in remain. These people acquire domain knowledge, "sometimes straight wisdom", making the solution -- its merits, design rationale, tradeoffs, and lessons learned -- more opaque to newcomers. Those newcomers find that existing solutions don't appear to do what they need at the level of complexity that they can grok. The cycle repeats.

Re:Jolt – JSON to JSON transformation by jpatokal on Wednesday November 23, 2016 @02:54PM (#13025330)
> a very simplistic serialization format with a minimal number of built-in structures.

It's already here: protobufs.

https://en.wikipedia.org/wiki/Protocol_Buffers

Re:Jolt – JSON to JSON transformation by arethuza on Wednesday November 23, 2016 @07:17AM (#13022021)
XSLT did have some nice bits - I still like XPath.

Edit: Just to be clear some of the most terrifying code I've ever seen (and possibly wrote) was in XSLT.

Re:Jolt – JSON to JSON transformation by benibela on Wednesday November 23, 2016 @11:22AM (#13023646)
If you like XPath and JSON, too, you can use my just updated Xidel (http://www.videlibri.de/xidel.html) to run XPath queries on JSON.

It can do transformations, where you write the output structure, and then pick the values from the input JSON. E.g. to transform from {"a": [{"b": 123}, {"b": 456}]} to {"c": [123,456]}, you could write an XPath expression {"c": a/b }

Re:Jolt – JSON to JSON transformation by manarth on Wednesday November 23, 2016 @07:58AM (#13022290)
Increase the terror further by adding Jsonx to the mix: http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com....
Re:Jolt – JSON to JSON transformation by arethuza on Wednesday November 23, 2016 @08:18AM (#13022402)
Isn't one of their examples invalid XML?

http://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com....

Shouldn't

    <json:string name="ficoScore"> > 640</json:string>
be

    <json:string name="ficoScore"> &gt; 640</json:string>
Re:Jolt – JSON to JSON transformation by manarth on Wednesday November 23, 2016 @08:36AM (#13022516)
It's one of the many peculiarities of XML: the "less-than" bracket is not allowed in text (because it could indicate the opening of a new XML tag), but the "greater-than" bracket does not need encoding when it's in text, because it's not ambiguous.
Re:Jolt – JSON to JSON transformation by arethuza on Wednesday November 23, 2016 @08:55AM (#13022627)
I'm pretty sure most XML handling libraries would use &gt; - never seen a raw > used in XML (though possible I might not have noticed).

Edit: Forgot to say thanks for pointing that out!

Re:Jolt – JSON to JSON transformation by benibela on Wednesday November 23, 2016 @01:29PM (#13024638)
Just check what the W3C says about writing a raw >:

> For characters such as > where XML defines a built-in entity but does not require its use in all circumstances, it is implementation-dependent whether the character is escaped.

(https://www.w3.org/TR/xslt-xquery-serialization-31/#serphase...)

Re:Jolt – JSON to JSON transformation by cowardlydragon on Wednesday November 23, 2016 @07:24AM (#13022069)
But but but it's FUNCTIONAL, and turing-complete!!!

// yes I understand the use cases of functional code

Re:Jolt – JSON to JSON transformation by DonHopkins on Wednesday November 23, 2016 @08:20AM (#13022418)
DSSSL [1], the SGML predecessor of XSLT and CSS, included a full blown built in Scheme interpreter, by the standard. That turned out to be a bit too much raw power and complexity for most people, so it never really caught on outside of the Boeing Airplane Maintenance Manual Publication Department and other big enterprisy SGML-loving organizations like that.

Microsoft's non-standard implementation of XSLT [2] is "out of the tarpit" Turing complete [3] because it lets you write handlers in JavaScript and other languages, to actually get some useful work done and call external librarys, without bending over backwards.

Of course Microsoft's non-standard extensions to XSLT aren't supported outside of Windows. But after using it, going back to plain XSLT was pretty frustrating to me, and in many cases it was easier just to forget about XSLT and write straight-up JavaScript code with XML parsing and XPath expressions.

There's nothing that XSLT can do that you can't easily implement in a JavaScript library. At this point it's better to start with JavaScript and forget about XSLT, instead trying to use XSLT, hitting a wall, realizing you want to ship on some platform other than Windows, then rewriting it all in Java, then realizing you want it to run in the browser, then rewriting it all in JavaScript, which you should have started with in the first place.

Another point for JavaScript: In the past decade, a whole hell of a lot more effort has been put into making JavaScript run fast, than making XSLT run fast.

[1] https://en.wikipedia.org/wiki/Document_Style_Semantics_and_S...

[2] https://msdn.microsoft.com/en-us/library/bb986124.aspx

[3] https://en.wikipedia.org/wiki/Turing_tarpit

Re:Jolt – JSON to JSON transformation by benibela on Wednesday November 23, 2016 @08:38AM (#13022529)
Isn't standard XSLT Turing complete anyways?

Even XPath 3 is Turing complete

Re:Jolt – JSON to JSON transformation by DonHopkins on Wednesday November 23, 2016 @08:51AM (#13022599)
The point is that XSLT is deeply stuck in the "Turing Tarpet": "Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy." -Alan Perlis https://en.wikipedia.org/wiki/Turing_tarpit

You would never want to write a JPEG decoder in XSLT, but the fact that it might be possible if you were willing to bend over backwards, write unmaintainable code that runs incredibly slowly and requires colossal amounts of memory, misses the point.

For getting real work done in the real world, where big companies have mountains of data to process, and are required to pay real money for electrical power and computing equipment, Microsoft decided to add the ability to call JavaScript, Visual Basic, and other scripts from XSLT templates.

Because it's just too hard to get anything of interest done in XSLT, and it's trivial and convenient in JavaScript, and there's probably already a library to do it.

Re:Jolt – JSON to JSON transformation by arethuza on Wednesday November 23, 2016 @09:01AM (#13022661)
"Because it's just too hard to get anything of interest done in XSLT"

I thought the point of XSLT wasn't really arbitrary processing but to transform XML into other formats (mostly, but not always, other forms of XML such as XSL-FO) - and it was reasonably good at that.

Re:Jolt – JSON to JSON transformation by DonHopkins on Wednesday November 23, 2016 @09:06AM (#13022683)
But who defines the line between "transform" and "arbitrary processing"? One language's transformation is another language's arbitrary processing. If XSLT doesn't know how to format dates for Eskimos, and I have an off-the-shelf JavaScript or .NET date formatting library that does, how does that make my requirements cross the line between "transformation" and "arbitrary processing"? It's a failure of XSLT to not offer Eskimo dates as a transformation (or a way to call a library that does that), not of me for requiring an Eskimo date transformation.

DSSSL was extensible in Scheme, so all transforms had access to arbitrary processing, by the standard.

XSLT was not extensible in any language, so Microsoft went ahead and made a non-standard extension that integrated JavaScript, Visual Basic, C#, and all other IScriptingEngine/.NET compatible languages, much in the same way standard HTML web browsers like Internet Explorer integrate scripting languages.

"The languages that you can use differ between the MSXML and .NET implementations. In MSXML (from version 2.0 on), you can use either VBScript or JavaScript. In .NET, however, you can use any .NET-supported language including C#, Visual Basic, JScript.NET, and even JScript, as before. The ability to use strongly typed languages like C# and Visual Basic .NET makes this an even more attractive opportunity." https://msdn.microsoft.com/en-us/library/bb986124.aspx

Lots of people got lots of work done with Microsoft's non-standard scriptable XSLT, because they believed it was just too hard to get anything of interest done in standard non-scriptable XSLT, but worth paying the price of being locked into Microsoft's ecosystem.

I think it's better to forget about XSLT entirely, because it's a long lost cause, weak and ineffective at what it was designed, and use JavaScript and libraries like XPath to transform XML (and that goes squared for JSON), to reap the advantages of the latest JavaScript engines and libraries.

If Jolt is meant to be a language-independent standard, there should be a definitive reference implementation in JavaScript.

(in reply to arethuza's reply:)

Yes, I'd rather have all the great things you can do in domain specific languages like XSLT (like XPath, CSS, RegEx, etc) packaged up into nice libraries I can call from JavaScript, instead of being big complex hybrid frameworks that call my little snippets of JavaScript, like Microsoft's extended XSLT.

Re:Jolt – JSON to JSON transformation by arethuza on Wednesday November 23, 2016 @09:36AM (#13022898)
Och yes - I was definitely not recommending preserving XSLT (or indeed a JSON based equivalent). I always found that special purpose languages like XSLT were quite good but that for most tasks general purpose programming languages were far better so - the times that recursive pattern matched templates in XSLT made sense were few and far between.
Re:Jolt – JSON to JSON transformation by benibela on Wednesday November 23, 2016 @11:08AM (#13023534)
But it is not deeply in the tar-pit.

Here is an raytracer written in XQuery: https://dev.w3.org/cvsweb/2011/QT3-test-suite/app/Demos/rayt... (imported modules are in the directory above) looks quite maintainable to me.

XQuery is pretty much XPath-with-functions, and XPath 3 has anonymous functions, so you can directly translate it to XPath 3 by replacing `declare function foo(..` with `let $foo := function(...`

Once it is XPath, you can put everything in a xsl:value-of tag.

Re:Jolt – JSON to JSON transformation by DonHopkins on Wednesday November 23, 2016 @11:36AM (#13023731)
I'd prefer to be on a mountain top far away from any tarpit, than only partially submerged in a tarpit.

The mountain tops are modern JavaScript engines. How much work have Mozilla, Google, Apple, IBM and many other big organizations and excellent engineers poured into making XSLT run fast and efficiently in the last decade, compared to how much has been applied to JavaScript? Can you cite any interesting SIGPLAN articles about XSLT optimization? Have any commercial games shipped using a ray tracer implemented in XSLT?

Re:Jolt – JSON to JSON transformation by mhd on Wednesday November 23, 2016 @09:41AM (#13022934)
As if we're not already doing that. JSON configs are quite popular (e.g. package.json), and it's not really a format made for human consumption.

We all cried about the abuse of XML and its inherent foibles, but I seriously doubt that replacing that with JSON and Markdown was the best choice.

Re:Jolt – JSON to JSON transformation by tonyedgecombe on Wednesday November 23, 2016 @07:16AM (#13022014)
Seems so, just like how we are turning JavaScript into Java.
Re:Jolt – JSON to JSON transformation by martin-adams on Wednesday November 23, 2016 @07:25AM (#13022074)
But without a standard library
Re:Jolt – JSON to JSON transformation by cowardlydragon on Wednesday November 23, 2016 @07:23AM (#13022066)
XPath is fantastic, although not everything is relevant to non-XML docs.

But in general, JSON, XML, S-expressions, yaml, HTML, and many data interchange programs are all 95% the same hierarchical data structure.

I wanted to make a cross-dataformat XPath subset that would work on all of them, and even allow a format that mixed the formats as needed in a MIME-esque document format, but the parser... oh god the parser.

This guy's project was like the initial steps of mine: use whatever XML tools exist, but that kills streaming processing and doesn't pass cursory smell tests...

I like this though, it is food for thought.

Re:Jolt – JSON to JSON transformation by zeveb on Wednesday November 23, 2016 @11:49AM (#13023829)
> But in general, JSON, XML, S-expressions, yaml, HTML, and many data interchange programs are all 95% the same hierarchical data structure.

S-expressions can also represent graphs. I have a vague recollection that YAML can as well.

An S-expression example:

    (nodes #1=(a #2=(b #1# #3=(c #2# #3#)) #3#) #2# #3#)
Re:Jolt – JSON to JSON transformation by mmastrac on Wednesday November 23, 2016 @12:51PM (#13024292)
Firefox's Spidermonkey used to support a similar notation for JS objects. I can't remember when it was removed, but it was pretty obscure:

http://blog.notdot.net/2006/9/Serializing-JavaScript-objects...

http://philogb.github.io/blog/2009/03/24/sharp-variables/

https://developer.mozilla.org/en-US/docs/Archive/Web/Sharp_v...

Re:Jolt – JSON to JSON transformation by junke on Wednesday November 23, 2016 @12:16PM (#13024020)
Worth noting that you can both read and print such forms.
Re:Jolt – JSON to JSON transformation by zeveb on Wednesday November 23, 2016 @04:52PM (#13026065)
> Worth noting that you can both read and print such forms.

That's actually how I checked that I had the syntax right: I wrote:

    (let ((*print-circle* t) (*print-case* :downcase)
      (print '(nodes #1=(a #2=(b #1# #3=(c #2# #3#)) #3#) #2# #3#))
      nil)
Thus proving that reading & printing work well.

Seriously, Lisp is an awesome language.