This is what you call an "internal DSL", a DSL implemented as a valid subset of another language. A DSL that is its own language requiring its own interpreter is sometimes called an "external DSL".

http://en.wikipedia.org/wiki/Domain-specific_language#Usage_...

As a counterexample, Yesod (A Rails-like web framework for Haskell) uses (very appropriate IMO) external DSLs for defining URL routing and the data model. These are not runtime config, they are parsed into Haskell code, typechecked and compiled ahead of time:

models: https://github.com/snoyberg/haskellers/blob/master/config/mo...

routes: https://github.com/snoyberg/haskellers/blob/master/config/ro...

Maybe i'm a fundamentalist, but for me an internal DSL is more than what you describe. It is, for example, a syntax transformer (à la Racket) which is a new "keyword"-like thing. Valid in the host language, yes, but perhaps doing arbitrarily complex compilation-like steps before the final execution phase. IANAQCSY, though.

I especially dislike overuse of the DSL term because many Ruby type of DSLs are not compiled and are thus act like stubs that can only resolve at runtime. Nobody would call SOAP bindings generated by a code generator a DSL. The only distinction I am seeing in practice now is that a DSL does not mean you have to learn a lick of the foundational language to be productive in it. This would be different from libraries like Scalding in Scala for doing queries where they're just sugar and more like an ORM. But nobody calls an ORM a DSL despite changing a lot of semantics used to reason about the underlying system.

As far as I see it, you can certainly call your API / library a DSL when you have to write your own compiler rules for it in some capacity. Chef and Puppet code are not DSLs in this respect (especially Chef - the files have .rb extensions, gee) while SQL and CSS would count as DSLs free from being tied to a specific foundation language.

So, in your mind compilation is what distinguishes a DSL from... something else?

The need to perform a platform-independent parse + compile is a large part of my personal concept of a DSL. This is not the same thing as writing shims and grafting new methods onto objects to mutate aspects of the language you're writing in already while tying yourself to, for example, variable scoping rules that is inherited from an implementor language. In fact, I've seen some people describe what amounts to a "Fluent API" as a DSL. Fluent APIs are great and help with usability of an API considerably, but I don't think of jQuery or Scalding as DSLs, just libraries that meet all the rules of their implementation languages. Perhaps some people simply don't care and just think of a DSL as a subset of a language regardless of implementation.

If I want to be a bit more specific, I think a DSL needs to be independent in syntax and semantics from any implementation environment or platform and that by leveraging the compiler of your implementation language that it limits the "language" aspect of the DSL. I believe most people are concerned about the "DS" aspect and in this respect that reduces a DSL to being just a higher-level API - an API is not necessarily Turing complete, after all, and while some APIs require some parsing to use, they are still.

An embedded language like how Lua is used oftentimes is not a DSL to me because it is general purpose and is perfectly fine to implement another DSL - a DSL to me should also imply that you can't use it to implement another arbitrary DSL. This will blur as what used to be DSLs add features that make them Turing complete (lots of crazy things possible with SQL and CSS these days).

I don't like to be a terminology Nazi or something, but I have no real rational explanation for why "DSL" in practice rubs me funny.

Huh, yeah. I don't really enjoy semantic arguments, either, but I think you've got a lot of great points.

There is something really genuinely different about a "DSL" which has reached the status of being totally externalized. On one hand you're now forced to more completely define the semantics/syntax from scratch (you can't bootstrap, though I think to some degree that 'definition by implementation' here is a big fallacy) but then on the other you're also facing a much greater task if your primary end goal is to just better focus on the domain.

In my opinion, DSLs are just languages with an intent to hit a target domain. External DSLs, as I call what you call DSLs at large, are one mechanism for doing that. Internal DSLs, which are not really distinct from APIs + their conventions, have the interesting property, however, of arising in layers. This aids composition and speed to develop.

But they absolutely suffer from being compromised on semantics and syntax due to their embedding. And before a Lisper comes by and talks about macros, I think the tradeoff is fundamental. The more you bear on the "underlying" language in your DSL the more you must compromise.

Genuinely, a very high-value DSL ought to be externalized, I feel.

It's pretty common to call a DSL "internal" when it works within the syntax of an existing language, as opposed to an external DSL that is a separate language. I'm not sure saying "but it's not really a language" helps us stay clear on what's going on.

So what's the difference between an internal DSL and any other library?

Usually you call something an internal DSL when it reads like its own language. The reason that Ruby lends itself to internal DSLs is how optional a lot of its punctuation is, especially parens around method parameters, making it easier to write code that looks like a variant of English. A good example of this is Rake.

I still don't get what 'reads like its own language' really means though. Rake is still Ruby syntax and semantics. Rake just adds new methods - they're still Ruby methods and calling them reads like any other Ruby method using the same parsing rules.

I'm playing the fool here - I do know what an internal DSL is but I think the term is entirely meaningless when you really think about it.

Lisps let you define the semantics and syntax of a form with macros. I would consider the loop macro in CL to have an internal DSL.

--edited

And another example of this, but not using macros, would be Rebol's parse which is a top-down parsing dialect (internal DSL) within Rebol.

ref: http://en.wikipedia.org/wiki/Rebol#parse

shrug I just didn't like the attempt at language policing, because it seemed pointless. This, however, is a good question that I don't know the answer to.

I think the difference becomes clear when you consider that a language syntax is not just source code. Raw, text source code is just the means by which humans express utilizing the syntax of the language. Once that source code file is tokenized and parsed, it's no longer tied to whatever that text was. It becomes an abstract object in memory that either represents a process (a series of machine instructions) or is fed to a process to be manipulated and processed (i.e. interpreted scripting languages).

For example, this blog post walks through McCarthy's original specification for Scheme: http://programmingpraxis.com/2011/11/01/rip-john-mccarthy/

    apply[fn;x;a] =
         [atom[fn] → [eq[fn;CAR] → caar[x];
                     eq[fn;CDR] → cdar[x];
                     eq[fn;CONS] → cons[car[x];cadr[x]];
                     eq[fn;ATOM] → atom[car[x]];
                     eq[fn;EQ] → eq[car[x];cadr[x]];
                     T → apply[eval[fn;a];x;a]];
         eq[car[fn];LAMBDA] → eval[caddr[fn]; pairlis[cadr[fn];x;a]];
         eq[car[fn];LABEL] → apply[caddr[fn];x;cons[cons[cadr[fn];caddr[fn]];a]]]

This is a type of source code. Notice it even includes non-ASCII characters. But this code isn't executable as it is. It has to first be translated into a program, somehow. See the first reply at the original link, which I will not reproduce here, for brevity.

It is a Scheme interpreter. The call to `evaluate` at the end of each test function is where the program gets interpreted. No, it is not a Scheme source file, but it is Scheme syntax, written in Python.

Similarly, LINQ to SQL is a SQL syntax written in C#. Your LINQ expressions are C# code that construct objects that will be interpreted in a SQL context. LINQ to Objects, on the other hand, is NOT a SQL syntax, it's just a library. Even though the function calls look exactly the same, because the C# code you write defines objects that then execute in a C# context.

Of course, that's not to say most "DSL"s haven't misused the term "language". I see it applied a lot to interfaces of chainable method calls. It isn't a new language, it's an exploitation of the existing language. The code still executes in the host language's context, it does not define its own contexts or semantics in any way.

Look at Emscripten. Is it C because that was what the code was written as, or is it JavaScript because that is what the code gets translated to? It's C, because it creates an execution context within the JavaScript code to recreate C semantics.

Look at this toy language I once wrote (https://github.com/capnmidnight/betty-boop/blob/master/loosp...) and a Pong game that uses it (https://github.com/capnmidnight/betty-boop/blob/master/pong....). Is the Pong game Scheme because that's what it was written as, or is it JavaScript? It's JavaScript, because though the code looks like Scheme, it executes like JavaScript, it's just a different syntax that gets translated away.

I'm not sure calling it a language does, either.

Yes, it's pretty common to call it that. Mainly because it's pretty common for programmers to be undereducated, so they misapply terms or unnecessarily reinvent terminology.

Let's call the "smart interface with domain-specific verbs" a domain-specific interface, it would be just fine. But calling it a language is simply a term abuse.

Before you start throwing insults, what exactly is a language to you? I'm pretty confident that determining the difference between a language and an API is not so easy a thing to do.

A language has syntax and semantics; the combination of domain specific semantics with general purpose syntax is a domain-specific language.

Of course, with host languages that support user-defined syntax rules, an internal DSL may also have specialized syntax.

The cumbersome term you invented is not an accurate replacement for internal DSL.

> it's pretty common for programmers to be undereducated...

[citation needed]

Not that I think it's an adequate citation, but there is an essay from Dijkstra on the front page of HN right now where he says essentially the same thing: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...

I've seen the distinction between internal and external DSL in academia as well.