News for hackers. Stuff that matters.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by haberman on Thursday June 14, 2012 @04:48PM (#4113445)
> JSON would do nicely as well.

Yep, some friends of mine did this with JSON, but didn't make the schema explicit like I mean to: https://github.com/benbernard/RecordStream

> Better yet, unify the shell with a virtual machine that is used to implement the OS, and have everything available as 1st class Objects.

Please no. This is the Microsoft PowerShell approach, where everything is a .NET object. Once you start dictating representations of objects, you are dictating far too much about the implementation of individual pipeline nodes.

Re:Rob Pike: "Current Unix tools are weakened by the built-i... by neutronicus on Thursday June 14, 2012 @06:28PM (#4113863)
How can your tools all accept the same kind of structured data without dictating its representation? I don't get it.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by haberman on Thursday June 14, 2012 @07:12PM (#4114056)
When I talk about a "representation," I mean an in-memory format. For example, the "representation" of an HTML tree is the DOM.

Yes, you have to agree on a serialization format (JSON, Protocol Buffers, etc), but that's not the same thing. From a serialization format you can represent the data however you see fit in your process. For example, a C++ user might represent a string as a std::string object whereas a Python user would represent it as a native Python string.

The VM-based approach (like PowerShell) defines an in-memory tree representation, namely .NET objects. This means that you can't really interoperate with this stack unless you use .NET too, since you don't have an easy way of converting .NET objects to your own objects.

Re:Rob Pike: "Current Unix tools are weakened by the built-i... by akkartik on Thursday June 14, 2012 @08:24PM (#4114304)
Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by haberman on Friday June 15, 2012 @02:49AM (#4115377)
It's true that many Protocol Buffer libraries include representations, but these are for convenience; Protocol Buffers are defined in terms of their serialization format and schema.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by akkartik on Friday June 15, 2012 @04:15AM (#4115543)
I don't follow. Isn't schema the same as representation? It's the equivalent of the DTD for an XML document.

To be super concrete, I can't read a file containing protos without knowing their type, what fields they contain, etc. I can however read JSON just fine without knowing the precise schema being encoded.

Re:Rob Pike: "Current Unix tools are weakened by the built-i... by Evbn on Friday June 15, 2012 @02:36AM (#4115345)
Schema isn't representation the way haberman is using it. He means something like implementation, which protobuf has in many languages in various VMs and can be easily ported to more.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by stcredzero on Thursday June 14, 2012 @07:46PM (#4114178)
Yes, but you could also just use IronPython and script in Python.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by haberman on Thursday June 14, 2012 @08:21PM (#4114297)
The more decisions you force on your users, the more reasons you give them to choose some other technology, and the less future-proof you are. For example, .NET is only really usable on Windows.
Re:Rob Pike: "Current Unix tools are weakened by the built-i... by jcmhn on Thursday June 14, 2012 @06:56PM (#4113983)
Allow a shell variable to control output record and field separators. Default to space and newline if nothing is specified.

ORS=: OFS=, ls -l # rw-r--r--:1:uname:gname:1025:Jun 1:somefile.txt,...,...

Add a specifier for dates, usernames, groupnames, etc. DSF="%Y-%m-%d" ls -l

Re:Rob Pike: "Current Unix tools are weakened by the built-i... by nwmcsween on Thursday June 14, 2012 @05:30PM (#4113601)
Why does the protocol define the objects? Shouldn't the object metadata should have enough information to transform the output? E.g. object is a URL vs protocol is a URL