If you want to increase your productivity in orders of magnitude, if you want to be able to complete projects on your own that otherwise large teams would struggle to do, you must learn Lisp or any other meta-language suitable for DSL construction. Luckily, now there is quite a few of them, so if you don't like Lisp in particular there is a number of alternatives available.
I've seen this claim a few times in this thread. What makes this possible? If you can increase productivity so drastically, why isn't everyone using it all the time? It seems utterly irrational not to.
Take a look at what kind of power DSLs can give you:
http://www.moserware.com/2008/04/towards-moores-law-software...
If you use this methodology consistently, all of your code (including the DSL compilers themselves) is just as readable as this. Imagine, all your code reads as just an English specification of a problem you're solving, with some tables and diagrams where needed.
> why isn't everyone using it all the time?
I do not have an answer to this question.
Yet, given that I never managed to convince anyone who was not already doing this stuff that macro-based DSLs are a way to go, apparently, there is a huge cultural problem and a huge stigma attached to macros (likely because of the C preprocessor and some really bad Lisp examples - think of the stuff like LOOP macro).
I personally use this approach exclusively, and I cannot understand those who prefer to write 1000x more code.
In my experience, most people who are so in love with macros/DSL's work on solo projects.
Are you working on a team? If yes, ask your coworkers what they think about your macros and it should be more obvious to you why macros/DSL's should be used much more sparingly than what you recommend.
> I cannot understand those who prefer to write 1000x more code.
You're setting up a false dichotomy. It's possible to not use macros and still keep the amount of code down to a reasonable level.
Yes, but not because it's hard to cooperate when you use DSLs (it's in fact far easier to work in a team this way). It's more because scale of the projects shrink to such an extend that you don't need a team. One person can do with macros an equivalent of what a team of many people would do in a much longer time without macros.
> If yes, ask your coworkers what they think about your macros and it should be more obvious to you why macros/DSL's should be used much more sparingly than what you recommend.
Yes, I work in a team, and most often worked in teams. Yes, DSLs were always the greatest performance boost available. No, your beliefs about DSLs being somehow incompatible with team work are entirely wrong and unfounded.
> It's possible to not use macros and still keep the amount of code down to a reasonable level.
Mind naming a single viable alternative?
Most of the links in that article are dead.
That's exactly what's wrong with this thoroughly broken industry. They should not have learned this awful religion in the first place. Now it's much harder to unlearn and re-learn from scratch than learn it without ever being exposed to the wrong ways.
This methodology is far, far easier than anything OOP (the latter cannot even be defined comprehensively), and if you learn it first you can become productive much quicker than if you go the typical OO way.
And the most important feature of this methodology is that it's far more accessible. You don't need to be smart to use it. No need to memorise dozens of "design patterns" and all that. No need to invent complex twisted designs, trying to cram as many patterns as possible into a trivial task. DSL-based approach is very straightforward and mechanical, and can be executed by practically anyone who managed to learn how to read and write (i.e., around 95% of the population).
I mostly agree with your comments, but here I think you're underestimating the difficulty of creating a good DSL way too much.
It's like saying that anyone who managed to learn to read and write can create Esperanto.
Also, being able to read and write does not guarantee the ability of clearly expressing one's thoughts. Without this crucial skill inventing a DSL is an exercise in frustration, both for the original author and later for the poor users of such a DSL.
And one more thing: you can create (internal) DSLs in almost all existing languages, by (ab)using their syntax and semantics. Smalltalk and Ruby are quite happy with their many DSLs, despite being an OO language.
As for GP:
> How would you suggest a typical run-of-the-mill object-oriented programmer go about learning such a methodology?
See Avail: https://klibert.pl/output/avail-and-articulate-programming.h...
Then learn Scala, Ruby, Elixir, Racket, Smalltalk, F#, Elm, Haskell. They all make use of DSLs and contain many examples of well-designed eDSLs. After that, go read a bit on language design and compiler construction and you're done.
I'm doing this for living, and I can't stop wondering at how laughably easy it is.
The method is very simple:
* describe the problem in plain English (maybe with some diagrams)
* iterate it a few times until you have a syntax you think is unambiguous enough
* strip this syntax from all the sugar you just introduced in order to define an AST
* find DSLs in your toolbox that are potentially close to the one you're building and cherry-pick the necessary language components from them
* write a sequence of very simple transforms that would lower your source AST into a combination of the parts of the ASTs of the target DSLs of your choice
* Done. An efficient DSL compiler is ready, with a language designed as closely to your current view of the problem domain as possible.
* If you later find that your DSL is inadequate and your understanding of the domain was insufficient, than just start over again, this entire process is so cheap and simple that it does not really matter.
> ability of clearly expressing one's thoughts.
This is crucial indeed, but not having such a skill is a functional illiteracy. Current estimation of the functionally illiterate population in the first world countries is from 5 to 10%. The remaining numbers are sill much higher than the percentage of people who are capable of understanding OOP.
> you can create (internal) DSLs in almost all existing languages, by (ab)using their syntax and semantics.
But in order to do this you need to be smart (or smart-assed). It's a hack, and hacks are bad.
OTOH, macro-based DSL implementation is purely mechanical and does not involve any thinking at all.
(and sounds totally intuitive and obvious, not like a trick, but like a proper methodology, proper approach, never had this feeling of before)
thank you for info! will try to dig in
It reminds me of Esperanto a bit, it might have some nice ideas but it will never take off. I've yet to hear a succinct, practical argument as ot why I should take the dozens of hours to learn lisp or any of its derivatives as someone working as a non-academic.