Using a programming language is like speaking in common tongue to every other programmer who knows that language. And most languages are so similar that most programmers can communicate even between languages.
Having code that looks the same, regardless of what it's actually doing, allows everybody to be able to read it.
As much as I love DSLs, I Couldn't agree more.
>The problem with writing a DSL for every problem (and every programmer writing different DSLs) is that you end up with all these different languages
When writing in a DSL of my design, it is amazing. It feels like the program models exactly how I am thinking about it.
When modifying/maintaining a DSL made by someone other than me, often this can be horrifying, it feels as though this models my thoughts so poorly (and it does, it was designed to model their thoughts).
How many people have learned what MVC means from Rails? Or the term "routing" in the context of a webapp?
One caveat I'll give: the DSL must be well-conceived. Writing a DSL that is dissimilar to both the underlying language and the domain vernacular is obviously horrible.
To add to what you wrote, you have to also consider that all developers in one team have to agree on a common DSL beforehand, which is an investment of time in itself, and the fact that if there is an unusually big change in requirements mid-development (which, let's be honest, is not a rare event), you might have to go back and change your "pyramid of DSLs" to accommodate it, maintain compatibility and update all your colleagues on the new languages.
With a DSL, all that goes out the window. The syntax of the language is potentially unknown to you. The semantics are an even deeper mystery. You're in terra incognito, trying to understand the mental process of the developer of the DSL before you can even understand what the code written in the DSL intends to do.
The primary goal of programming languages is not to describe how to perform a complex task to a computer. It is to describe how to perform a complex task to other humans in a form that computer can also execute.
When I talk to someone about what happens in a system it seems like a DSL flows from that conversation.
"It is to describe how to perform a complex task to other humans in a form that computer can also execute."
I think a DSL does that much better than a set of API calls, particularly when you involve non-developers. That is the point of a DSL.
And statements like (paraphrasing the quote in the article)
> You shouldn't build an app in Forth. You should build a language to model your problem domain in forth and use it to build the app.
They just doesn't make sense to me. Isn't that the very definition of how to program?
In a Lisp, you can use macros to accomplish things that go way beyond defining a domain-specific library / API. A typical library is restricted to the capabilities of the language, but a DSL in Lisp can extend the syntax itself.
Want to alter the reader so you have a way to express normal code as remote computations? Go for it. Feel like adding new math operators to use matrices instead of scalars? Knock yourself out.
Common Lisp itself did exactly this when object-oriented programming came along; using nothing but macros and closures, the Common Lisp Object System (CLOS) was built.
"and here is the 1M LOC codebase, which was carefully crafted to avoid using any of the complicated frameworks; all you gotta know is the C standard library".
Plus one outdated sequence diagram.
//! \brief Transport diagnostic wrapper
class TransportDiagnosticWrapper {
We are doing web-testing in clojure, and in past year our little framework evolved into a DSL of a sorts.
On one hand it is great, we a have consturct like (with-client [name connection & body]), that behind the scenes sets up a virtual machine, registers it as a client of our web-service and destroys it after commands in body are done. I can't imagine we would be doing setup and teardown by hand.
On the other hand it took me a month to get into it (two weeks learning the language, two weeks messing around with the framework) ... and there are still lots of things only our lead-tester knows how to fix, after our devs decide to change something we were relying on for testing.
I am curious, did you already know the JVM? When I learned Clojure, I was also learning the JVM, and for me the JVM was much harder than Clojure. So I would say it took me 6 months to learn Clojure, but much of that time was the time it took to learn about the JVM and all the related Java weirdness.
Either way, 2 weeks is very impressive.
What really helped me that, we had "programing paradigm" course in college, where I learned haskell for 3 weeks, what was enough that anonymous functions, map, reduce and using lists for everything didn't feel new.
Take SQL, a nice DSL for working with relational databases. Yet people ended up writing ORMs (quite ugly creatures imho - something between an elevator connecting two layers, and as in real life elevators are a mess when too many people want to use them at the same time, and an intermediate pseudo-level, but they are useful) because the pain of climbing up and down the ladder connecting the "relational level" with the "OO layer". "Language layers" or DSLs are a powerful but dangerous abstraction because when they start using them people can't stop, they just make more and more layers like crazy! (The only other abstraction so powerful and so dangerous I can think of are objects - it's amazing how many got them completely wrong so many times in languages like C++ or Java... besides Ruby and Smalltalk I can't think of anything that got them "mostly right"... and Ruby seems to be the most DSL-able language in use too, though this is why I'd take Python over it any day - smart programmers tend to make everything like a freakin DSL in Ruby and and your mind ends up running up and down the ladders until you go insane when reading code).
A sane relational database API would let you compose and extend arbitrary queries, which has profound implications for the way you can write your program and abstract away repeated patterns.
[Arel](https://github.com/nkallen/arel) has the right idea in terms of API, but a poor implementation (lots of things are not truly composable where they should be). [ScalaQuery](http://www.scalaquery.org/) purports to have the right idea, but I haven't tried it, and it's very Scala-centric.
I've recently been digging hard-core into C++ after making such disparaging comments for years and have been rather surprised to see that it's not as terrible as I once thought -- It's actually quite good.
I agree however with your point about dangerous abstractions. DSLs when implemented with careful, thoughtful restraint are rather powerful. SQL being a really good example. There's nothing stopping anyone from writing their database queries by directly sifting through and filtering datums from the domain using the primitive functions and operations of their programming language of choice... but it's clear that SQL is much more powerful and lets us express such problems in a succinct way.
But just like programming languages, once introduced, a DSL is likely to be misunderstood and abused. The problem I believe lies not with the technique but with the users. A double-edged sword is a fine tool but will likely maim an inexperienced user. Which will probably lead them to writing DoubleEdgeSwordWrapperMachineGun classes and then someone else to write a DSL on top of that...
Not NEARLY as cool of an implementation as implementing TCP using ASCII art, but there's a command-line option parser that parses the help text to determine the command line options. [1] For example:
-- scale.lua
require 'lapp'
local args = lapp [[
Does some calculations
-o,--offset (default 0.0) Offset to add to scaled number
-s,--scale (number) Scaling factor
<number> (number ) Number to be scaled
]]
print(args.offset + args.scale * args.number)
[1] http://www.inf.puc-rio.br/~roberto/lpeg/ [3]
[2] http://kernel.org/pub/software/scm/git/docs/git-fast-import....
[3] I actually use the re module [4] as I find the results more readable, and only use the LPeg module when I need something that the re module can't provide.
lapp has been around for four years; docopt is (looking at git commits) less than a year old. It strikes me as a natural way to do things in Lua, which is why (I suspect) it was done in Lua earlier.
One thing that positively impressed me were coroutines though. You can do some very interesting DSL things with it.
I'd say it is the same with all good computer languages. You can express whatever you want in python and lisp, but following different grammar.
But designing a language is hard, including a DSL. Even for experts. e.g. Codd had two attempts at a language for relational algebra, but "mere mortals" couldn't use them. Mathematicians create new notations at the drop of a hat, giving their own definitions to common terms willy-nilly, but this can be an impediment to collaboration.
OTOH, there's an advantage of having many different attempts at a language, inspiring and cross-fertilising each other, til eventually settling on a near-optimal design (if everyone designs their own DSL, this doesn't happen). In practice, you get one group of people experimenting like crazy - academics, visionaries, hackers - and a huge majority that waits to see if something emerges that they can standardise on. Meanwhile, experiments continue. (This is basically the "technology adoption lifecycle".)
The ruby community develops new DSLs very quickly, in the form of frameworks. I get the impression the rapidity of change is challenging, but the improvements are worth it.
If Functional Programming was mainstream, everyone here would be raving about OOP and its benefits.
Ultimately what you want is to borrow the good ideas of different paradigms to create elegant and practical code.
To put it another way, OOP is a pretty nice hammer, but so many people have been using it to bash screws in (including ridiculous cases like concrete!) that others have reacted by using screwdrivers for everything (including nails). Pick the right tool for the job.
Ah yes, the ol "it's all relative" argument.
* Heterogeneity of object abstractions: in order of complexity, you use namedtuples, classes, and metaclasses. In Lisp you'd use the same kind of abstraction across complexity levels.
* Similarly, there are lambdas, plain functions, and bound methods, all for pretty much the same thing.
* Which do you use when? How hard is it to change? http://lukeplant.me.uk/blog/posts/djangos-cbvs-were-a-mistak...
* The way so many configuration variables in Django are strings, that just "coincidentally" point to a class, module, or function.
I've been using Python for 10 years, and Django since pre-1.0. My current startup is built on it. The parts start to show.
You can bend your statement to your language, or your language to your statement. All essays look the same, regardless of the statement being made. English, Chinese, Persian.
> you can bend your problem to your language
>The former is epitomized by Java and Python--all Java and Python code looks the same, regardless of what it's actually doing.
He was saying that python and java don't bend as languages towards the domain, that is that they bend the problem towards them.
>or your language to your problem.
>The latter is the domain of Lisp and Haskell and Forth and so on.
And that Lisp, Haskell and Forth are the languages that bend.
I'm unabashedly in favor of the latter.
Ultimately, it's easier to reason about some domain in that domain. I would love my code for any particular project to look very similar to what I would write on a whiteboard when solving the problem. If I'm solving something inherently mathematical, I want the code to resemble math notation. If I'm writing some parsing code, I want it to resemble a grammar. If I'm writing an interpreter, I want it to resemble the operational semantics of the language. This fits into more "practical" domains as well: if I'm writing a web app, I want the routing to look like, well, routing and the database code to look as close to the database structure as possible.
Any details in the implementation rather than the domain should be separated from the domain logic. If you're trying to make the domain logic mirror the domain itself, this is essentially inevitable. This makes your code more general and easier to debug. You can redirect your domain logic to a different backend, and it's much easier to figure out whether you have a problem in the implementation or the logic.
More generally, such a strict separation of concerns is extremely important. The difficulty of a problem increases exponentially with its complexity: all the parts affect each other. If you manage to neatly split a problem like this in two, each sub-problem will be less than half as difficult as the main problem. So splitting away the high-level logic from the IO and the memory management and the other implementation details is inherently valuable. In fact, this is valuable even if the sum of both sub-problems is a bit greater than the original problem.
The malleable language approach gives you a very natural way to split your problems up: one part is implementing the DSL, modifying the host language or writing a library--creating abstractions. The other part is actually using these abstractions. I find this sort of split to not only be very useful but also very natural; it mirrors how I usually think about things.
One very interesting recent development taking this idea to the extreme is OMeta. There your program is essentially a tower of DSLs, with the base language just being a simple way to specify the DSLs. This idea really is extreme: for example, the code for implementing TCP is actually the ASCII art diagrams describing it! [1]
Of course, all the usual suspects like Lisp and Haskell are also worth looking at. Embedded DSLs in these langauges can go quite far, and they have some of their own advantages to consider.
[1]: http://www.moserware.com/2008/04/towards-moores-law-software...