News for hackers. Stuff that matters.
Re:Japanese Grammar in EBNF notation by derefr on Sunday January 05, 2020 @12:04PM (#21962232)
Japanese is quite structured, in the sense that it doesn't form a very hairy AST data structure. But, as someone who knows both Japanese and a variety of programming languages, I've always seen the "seams" of Japanese a bit differently. I've always thought of Japanese as a stack-based language, like Forth.

A refresher on Forth: in Forth, you run a lexer over text input to get a stream of tokens; but you don't run a parser (or at least not much of one) over the tokens to get an AST. Instead, each lexeme—each "word" in Forth—self-describes as either a literal or a symbol representing a function call. All words the lexer encounters are immediately "run" using the runtime. Structured programming (like defining functions and then later calling them) is enabled by having the runtime itself be a finite-state machine, that can be put in different states by the execution of certain words, such that all words that are executed in the new state are executed to different effect (e.g. the word `[` will make all words up to a matching `]` have the execution semantics of pushing their symbolic representation onto the stack, sort of like a Lisp quote; the `]` itself then captures what's on the stack and builds a function, sort of like the `defmodule` macro does in Elixir.)

Analogously, in Japanese, most words are literals, that just push themselves onto the stack; while there are two types of "active" words: grammatical particles and verbs. Each particle is a construction function, which attempts to match and consume a certain shape of existing words on the stack, pushing back a tagged structure in its place; and then verbs consume a particular set of tagged structures from the stack (varying by verb), in any order, leaving on the stack anything they didn't expect, and pushing back a representation of the structured meaning of the sentence-as-a-whole up to that point.

I think this is well-represented by showing how Japanese does quotation:

上司は「私は彼が好きじゃない!」と言った。

To transliterate and rewrite this in a Forth-like grammar, with grammatical particles as lower-case keyword symbols, verbs as upper-case bare symbols, and everything else as quoted literal words:

"Boss" :subject [ "I" :subject "him" :referent DISLIKE NEGATION ] :cons SAID.

Note that, in this particular case, the brackets are sugar here: the sentence would have the same semantics without them. (They're helpful to visually disambiguate where you should mentally backtrack to, but it's clear by the fact that there's two :subject-particles that the inner verb DISLIKE is only going to capture the last-pushed one, leaving the previously-pushed one on the stack for SAID to later consume.)

One could also describe this as what a shift-reduce parser does internally, but with the reduction edges triggered as explicit command-words in the input lexeme stream, rather than being triggered implicitly by non-conflicting pattern-matches.

And this is also, as it happens, the core of any pure-functional abstract-machine bytecode ISA (e.g. Erlang's BEAM bytecode ISA.) You've got ops that push literals; ops that take patterns of stuff off the stack and push back new product-types containing those same things; and ops for calls to (maybe-built-in) functions named by a pushed symbol term.

Re:Japanese Grammar in EBNF notation by glandium on Sunday January 05, 2020 @07:37PM (#21965433)
Nice analogy, I think.

I'll just point out two things odd with your sentence example though:

- はvs.が is a difficult topic, but I think 上司が would be more appropriate

- you're supposed to use keigo when talking about a superior, not only when talking to them. So it should be 仰った(おっしゃった) rather than 言った.

Re:Japanese Grammar in EBNF notation by carapace on Sunday January 05, 2020 @12:16PM (#21962336)
Very nice description of Forth BTW.