Great comment. And if a domain-specific-language (DSL) of really custom syntax is needed, an alternative is to use Racket (a Lisp-family language), which allows for creating DSLs of all kind. Racket is already used for designing your own programming language and trying it, so I'm surprised that the author did not mention suggest Racket as a test-bed for the new language.
> This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly.
Correct. For example, most of the common systems used today do use a 'code generator' internally: Every software that uses an ORM library (i.e. Hibernate), is already using a sort of domain-specific-language (i.e. HQL) that, via a code generator, translates to another language (i.e. SQL). So most major platforms (Java, .NET, etc) already have one or many popular libraries that do implement some sort of (limited) DSL and code generator.
I think the aim of the article was to create a new compiler from the ground-up. I think the correct choice for syntax is something that can't be boiled down to three pieces of advice or one article. It is a topic that would require very deep discussion.
Most of my interest stems from this excellent project in Racket that's shaping up to be quite something:
https://github.com/lexi-lambda/hackett
It uses the "type systems as macros" paper's approach to create a Haskell-like language in Racket.
(btw, I think you meant e.g., not i.e.)
Yes! You can override the reader so that it parses anything, and there are modules for writing grammers[1][2]. Still, it is usually much easier to just use the built in s-expression parser, and if you're writing a lot of lisp, you probably don't mind s-expressions anyways :)
Hackett has interested me too, I hope that progress can continue to be made, it would be a nice alternative to Typed Racket (which is great, but very complex).
I'm working with the author of the "Type Systems as Macros" paper right now, we're working on implementing a linear language (along the lines of Rust) using Racket's macro facilities and the turnstile[3] package. Being able to embed arbitrary type systems in Racket really shows the insane power of the macro system.
[1] http://docs.racket-lang.org/parser-tools/LALR_1__Parsers.htm...
If you're designing a language as a learning exercise, or a domain-specific one, my advice is to go right ahead, using a Lisp variant. If you don't know one, now is the time to learn one.
If you're developing a general-purpose language, ask yourself how your language will be better than, or at least different to, every general purpose language in current use. Be ambitious. As languages belong to families (e.g. Algol variants, Lisp variants, functional languages, visual dataflow languages, etc.), you should aim for your language to be the best in its family.
Your language should also be one you'd use yourself.
I'm developing Full Metal Jacket (http://web.onetel.com/~hibou/fmj/FMJ.html). It's very different from almost everything else, so I'm not expecting popularity overnight.
The Dr Dobb's article is way too conventional for my requirements. My language doesn't require any parsing. Syntax, type, and memory errors can't happen. The ideal target machine would just run the code directly without any need for compilation, though I will at some point need to cross-compile it onto the von Neumann architecture, and for that dataflow analysis is unnecessary, making optimal object code simpler to generate.
This is the direction I'd like to see computing go.
This "correction" is mischaracterizing Walter Bright's article. He's writing about industrial "professional languages" and that context is made clear multiple ways: 1) he's referencing his D Language and presumably languages like it, 2) desire for the compiler to display quality error messages, 3) a good runtime library. The comparables to D Language would be Java/C#/Golang/Rust. All those languages took years by a team of people to get to v1.0.
> we have seen code generators deployed regularly.
But no professional language's (like D's) canonical compiler I know of is the direct output of yacc or ANTLR.
It seems like you have made reasonable and correct statements about programming languages -- for someone else's article -- but not specifically Mr. Bright's. The context really matters here.
Embedded languages can have all the features you've named. Although I do agree that usually they piggyback on an existing bytes->bytecode system. But you mention YACC and ANTLR as the mechanisms for doing this whereas someone using Lisp or Haskell need not use an external parser, which lowers the cost.
I can implement a C interpreter in Haskell or Lisp. We routinely make language extensions to support all sorts of things, and we do it with excellent error messages and supportive runtime libraries.
Maybe if the author had titled the essay, "So you want to build a runtime and optimizer then attach a lexer and parser to the top" I'd have rephrased. But he didn't, and I took the opportunity to make my point.
> It seems like you have made reasonable and correct statements about programming languages -- for someone else's article -- but not specifically Mr. Bright's. The context really matters here.
Maybe if people want to make nuanced points they should stop using clickbaity titles and leaving major tenants unsaid save for "contextual" queues applied ad hoc by others in 3rd party forums?
Pretty much impossible. If you don't leave your original language, you don't use domain specific language, you just have general purpose language with domain specific API.
Always preferred more productive ways to prototype programming languages.
We need no-nonsense lexer and parser generators, that are easy to use, easy to deploy, not too hard to implement, and generate fast parsers. My own studies¹ suggest it can be done. Unfortunately, I'm busy earning a salary right now.
My professor was unimpressed.
You can leave away higher-kinded types if you leave away types altogether, but I don’t think anyone has had a practical benefit from a monadic effects system or transformer stack in e.g. Javascript. I’m not even sure the plain »monad« concept even exists as an abstraction instead of in docstrings.
I don't really see what you're saying here. I didn't mean to equivocate the structures except in terms of their "alien-ness".
But Monads are "special", in that literally everyone uses them but almost never sees them from the "outside" unless they're using a language that offers techniques to model them. Writing Javascript? You're in a monad with helper tools of a specific type. Writing C? Same deal, different shapes.
Lisp Macros do something different (and I alluded to this with the "meta-syntactic" mention), but similarly change the thought process from "how do I write this logic" to "how do I describe this logic".
> . I’m not even sure the plain »monad« concept even exists as an abstraction instead of in docstrings.
Since they are annihilated in the Haskell runtime as well that shouldn't be terribly surprising. It's a modeling technique to build programs algebraically. It's like asking if Generics exist at runtime. They do inasmuch as they are relevant to the interpretation of the program.
The »plain monad« part was directed at Javascript, not Haskell. Haskell, like JS, can work perfectly fine without monads, but it still works perfectly fine with them ;-)
main :: [Response] -> [Request]
https://themonadreader.files.wordpress.com/2014/04/fizzbuzz....
http://dave.fayr.am/posts/2012-10-4-finding-fizzbuzz.html
This uses a slightly different set of abstractions with the same problem.
But perhaps more specifically, when you select a monad stack (group of effects) to compose to solve a problem you're building the features of the language and its effects. And if you use the "final tagless" or "Free" approaches you're doing that even more directly.
type Rand g = RandT g Identity
newtype RandT g m a = ...
-- with
runRandT :: (Monad m, RandomGen g) => RandT g m a -> g -> m (a, g)
newtype MyType = Rand StdGen
-- now you can define
intGreaterThan :: Integer -> MyType Integer
intGreaterThan = ...
randomName :: Integer -> MyType String
randomName length = ...
-- an finally convine the two above
nRandomNames :: Integer -> Integer -> MyType [String]
nRandomNames length n
| n <=0 = return []
| otherwise = do
tailNames <- nRandomNames (n-1)
headName <- randomName length
return (headName:tailNames)
[0]hackage.haskell.org/package/MonadRandom-0.3/docs/Control-Monad-Random.html
#include <stdio.h>
int
main(int argc, char **argv)
{
double i, s;
s = 0;
for (i = 1; i < 100000000; i++)
s += 1/i;
printf("Almost infinity is %g\n", s);
}
Lennarts-Computer% gcc -O3 inf.c -o inf
Lennarts-Computer% time ./inf
Almost infinity is 18.9979
1.585u 0.009s 0:01.62 97.5% 0+0k 0+0io 0pf+0w
import BASIC
main = runBASIC' $ do
10 LET I =: 1
20 LET S =: 0
30 LET S =: S + 1/I
40 LET I =: I + 1
50 IF I <> 100000000 THEN 30
60 PRINT "Almost infinity is"
70 PRINT S
80 END
Lennarts-Computer% ghc --make Main.hs
[4 of 4] Compiling Main ( Main.hs, Main.o )
Linking Main ...
Lennarts-Computer% ./Main
Almost infinity is
18.9979
CPU time: 1.57s
LBB1_1: ## _L4
movsd LCPI1_0, %xmm2
movapd %xmm1, %xmm3
addsd %xmm2, %xmm3
ucomisd LCPI1_1, %xmm3
divsd %xmm1, %xmm2
addsd %xmm2, %xmm0
movapd %xmm3, %xmm1
jne LBB1_1 ## _L4
OMG that is astonishing
Variable capture is a problem with lisp macros that lisp and clojure programmers have tools and strategies to manage.
A number or languages use hygienic macros to avoid the variable capture problem: Scheme, Racket, Dylan, Elixir, Rust, Julia & Perl 6 (which has hygienic AND unhygienic macros)
Racket has more advanced macro facilities specifically aimed at making new languages languages and DSLs. Syntax-parse is very interesting in how it addresses the problem of error reporting with macros.
Just want to add that Racket, and most Schemes also support both hygienic and unhygienic macros, through defmacro. [0][1]
Though, their use is heavily discouraged. (And it may be worth pointing out that Guile's defmacro is actually implemented using syntax-case, which is hygienic most of the time, but flexible enough to let you do madcap things. [2]).
[0] https://docs.racket-lang.org/compatibility/defmacro.html
[1] https://www.gnu.org/software/guile/manual/html_node/Defmacro...
Clojure picks the worst of all worlds by making reduced-power backtick macros, not stack tracing the macro output on failures, and then having a culture that discourages the use of macros.
These languages put you at the level of writing a language for each problem domain immediately and with a comprehensive and useful toolkit. These languages also do this while directly competing with all but the most carefully tuned of C++/C environments. Steel Bank Common Lisp and the Glasgow Haskell Compiler are good references here.
These languages are often considered strange because of this, and feel somewhat alien. But once you realize you're building a language to model your problems, suddenly tons of stuff makes more sense. Previously alien concepts like Macros and Monads are outside the typical language's experience precisely because the language authors created these contexts and put you inside them.
This article is sort of fundamentally wrong that it's difficult to write small, purpose built languages. It's not, and even outside the functional and homoiconic meta-syntactic world we have seen code generators deployed regularly. Successful products and libraries are build using these techniques all the time, and with a modern toolchain it delivers excellent results. It's just that other more restricted and guided approaches are often introduced earlier in people's learning curve and sets the expectations for them subsequently.