The other thing that bothers me is the empty 'Use cases' page. Scala macros might be a solution to a problem nobody has. A problem that people _do_ have is the breadth of Scala language, and macros will add to the conceptual dog pile.

1. The embedded language will be type checked at compile time. For example a regular expression macro could enforce that your regular expression has balanced parentheses in capture groups. This potentially lets you shift a class of errors from run-time to compile time.

2. The language will be parsed at compile time saving run-time cycles.

3. The host language can be extended and experimented with without adding new syntax. Features like this actually limit the growth of the "dog pile", since many language experiments features can be tested as macros first.

Some common DSL examples are regular expressions, parser generators, SQL / ORM generators and LINQ

Pls read slide 10 of this document - http://scalamacros.org/talks/2011-10-18-WhatsUp.pdf

--Proposed Code snippet with macro ---

class Queryable[T, Repr](query: Query) { macro def filter(p: T => Boolean): Repr = <[ val b = $this.newBuilder b.query = Filter($this.query, $reify(p)) b.result ]> } val products = db.products products.filter(p => p.startsWith("foo")).toList --

My takeaways : 1. adding LINQ support to Scala via macros 2. macros virtualize XXX, where XXX = vals, vars, defs, types, classes, traits. So macro XXX can be used in place of XXX.

The author also says "We can tailor macros to fit your particular use case". He mentions OR-mapping, and a Scala-to-javascript compiler that might end up using these macro invocations, among other things.

Another example would be about adding your own improvements to the time-system, improvements that are actually going to help you keep the code safe. Like the units of measure from F#.

However, adding powerful macros to a language that was not designed for macros and that is NOT simple ... I do think it is a bad idea. I'd be happy to see in Scala something like the code quotations from F#. Anything less is useless, anything more will make Scala way too complicated.

We've posted several compelling use cases enabled by macros (http://scalamacros.org/usecases/index.html) and added some more information to the main page that uncovers the inner beauty of the examples.

Btw, we realize the responsibility of adding a competely new feature to Scala, so every new macro-related idea is being weighed against the complexity it brings to the table. So far we've done away with several really interesting prospects in order to keep the overall proposal simple.

The other nice thing about these is that you don't need to lose type safety to use macros. That's really where the innovation is here.

I don't yet fully understand how this all works at the type-theoretic level. I'm imagining that expressions (as opposed to the values they represent) are thought of as belonging to higher kinded types. The macros then reject expression which aren't of the right kind.

I have to say, I am absolutely flawed by the comments so far about this. I think this is going to have an absolutely pivotal effect on future programming languages. This is how macros should always have been. I don't know of any other language which does this. It's a totally new area in language theory to explore.

They look very similar to Perl6 macros. See recent discussion here: http://news.ycombinator.com/item?id=3124920

If you want proof you only need to look at the error messages generated by most compilers. They probably make perfect sense to someone who is trying to debug the compiler, but they usually leave the consumer, ie the programmer using the compiler, somewhat confused.

Macros are bad because they redefine the language. The point of having a language in the first place is to make it easy to communicate. This is only possible if the semantics are understood. If you do good language design the semantics are clear. If you do sloppy language design people have to look things up all the time.

Now, if everyone gets to extend the language in arbitrary ways, you have suddenly unloaded a massive burden onto your programmers. It is no longer enough to say that you know language X. In order to understand the software you have to understand the specific dialects (plural) of language X -- because for a non-trivial project with lots of people extending the language in various ways, you will end up with multiple dialects.

When you can no longer bring up a piece of code in your editor and see what it does without having to reference dozens of other files that may modify the semantics of the language (sometimes in non-obvious ways), you are going to be a lot less productive.

I think Joshua Bloch was absolutely right when he said that one of the really good decisions of Java was to NOT have macros. He made some of the same points I made above.

Now I understand that people think macros can lead to elegant code, but that code only has value if other people can understand it. If you need to re-learn the language, in big or small ways, to understand it, its value is very limited.

And again: what excites language nuts and what has real value is not always the same thing.

It has been said that at any given point, every project implements their own programming language, usually as some kind of DSL, and they do it really really badly. I've seen this borne out in practice in EVERY commercial project I've worked on. I've seen it, I've lived, and I continue to live it every day. Macros are one of the ways out of this mess that we've created for ourselves. While a lot of macro research is still cutting edge, slowly different techniques are becoming more proven and starting to disseminate more and more into mainstream languages and runtimes. The problems you mention with macros have honestly already been dealt with in the past decade or two.

I've dealt with people like you at work before. Your kind are those that stare at any abstraction they don't understand with immediate suspicion and reject it without any real consideration. They refuse to learn new things or improve their programming abilities or toolset. While this in of itself isn't a crime, they hold the rest of the industry back by rejecting superior technologies as they come along because they refuse to take the time understand them.

Your kind and mine, sir, are in direct opposition.

On one hand you are saying that most projects you have worked on end up creating their own language, usually as some kind of DSL. Then you argue that the way to counter this is to ... give people more tools to craft DSLs.

So what are you trying to say?

PS: you seem to make a lot of assumptions about me based on ... what exactly? I don't believe we've met and whatever you are "in direct opposition" to is simply a straw man you have imagined.

Macros in Scala? I don't know, the language already seems quite complex without introducing custom syntax. But if they're more of an occasionally-used compiler extension for specific internal DSLs rather than a by-example use-them-everywhere, I don't see them hurting.

That being said, I'm quite enjoying taking a break from JVM hell and doing some exploratory programming in Racket. If special forms are well documented, widely used, and syntax checked, the extra mental overhead isn't burdensome.

I think asking language nuts (which isn't limited to those who write compilers, but also extends to practitioners with deep knowlede) to be aware of this is particularly important in that people look to them for enlightenment and advice -- and when they do harmful things it has a huge multiplier effect.

Some people from our team do have a painful experience with Lisp-style macros, so we try especially hard to find a minimally useful yet conceptually clean approach to compile-time metaprogramming (we are sure, though, that this sweet spot exists).

And that's why we stress the importance of feedback and appreciate it. Thanks for sharing your thoughts!

A macro, in its simplest form, is a function of type syntax -> syntax, or, a function that takes syntax as input and produces syntax as output. These functions are invoked at "compile time" as the compiler is processing your source code, and the resulting code is then compiled or interpreted along with the rest of the code. The way you write these functions, what these functions are capable of doing, and even the precise data type of syntax varies between the languages that implement macros.

The most powerful macro systems allow you to "extend" your language with new syntactic constructs and features. In this sense you can think of macros as a way for you to team up with the compiler to improve the language. Macros also allow you to generate repetitive code that would be tedious or error-prone to hand-write and maintain on your own.

The easiest way to understand good macro systems would be to learn about them from a Lisp-variant like Clojure, Scheme, or Common Lisp, as macros have typically been the domain of these languages. It is still an active area of language design research to try to effectively integrate a good macro system into non-Lisp languages. Scala's macro proposal is supposedly heavily inspired by a language called Nemerle, which runs on the .NET platform. I don't know much about Nemerle, but it appears to be innovating very well in this particular space.

One of the designers of Dylan tried to push this in his new language called PLOT (Programming Language for Old Timers). It has pythonic syntax and provids a full powerd macro-system. Unfortently there is no implmentation of PLOT just a discription: http://users.rcn.com/david-moon/PLOT/

They work by not evaluating expressions but treating them as blocks of unevaluated code. The blocks of code can then be manipulated, transformed, split apart and reassembled all using the full power of the language. You aren't manipulating the values of those expressions, but the code itself.

The simplest example of a macro is a function which does not evaluate its arguments until you specifically tell it to do so. But they can be arbitrarily more complex than that.

Lisp is very flexible because it has no syntax to speak of and code is data. So manipulating the code is the same as manipulating data. With languages like Scala that have syntax, macros have to work with either strings representing the language elements or objects which represent them (expression trees as they are called in the post). In other words, they work with some abstract version of the syntax of the language.

Observe (aaa bbb ccc). If you just look at it as a datastructure, it's a list of 3 elements. If you look at it as executable code, then aaa is the operator, which operates on bbb and ccc.

If aaa is that special kind of operator which is a macro, it can rearrange (aaa bbb ccc) to be something different, which is then executed. Anything. It replaces itself with something else. If you programmed in a language like that, you'd never have to wait for some language designer to create a new for() loop. Because you could easily write a loop which is then transformed into the usual for(i=0;...) loop.

The evaluation strategy for Lisp functions is to evaluate each of the inputs before making the function call. In the example above, (* 4 5) is evaluated first before the call to the function + even begins; once you have the intermediate form (+ 3 20) where all the parameters are fully-evaluated, the function call happens.

Why are macros important? Let's say that you want to write a short-circuit AND. You can't do that as a function. For example, (and false (/ 1 0)) will return false (never evaluating the right-hand expression) if you have short-circuit AND but it will raise an error (division by zero) if you don't. If that were a function, that call would (undesirably) fail. So what this means is that you can't write the short-circuiting AND as a function. You need something else that operates on expression forms, not values. In practice, it's a macro whose behavior is approximately like so:

(and <X> <Y>) => (if <X> <Y> false)

This is a transformation of code that happens during compilation, not a run-time function call.

It's worth noting that IF is not a function or a macro. It's a "special form" provided by the language. The evaluation of if-forms is not like the evaluation of function forms; some branches are never evaluated. It requires special-case handling.

That's one use for macros: writing control-flow primitives. There's a lot more to them as well: code generation to eliminate boiler-plate, DSL creation. Common Lisp, for example, has an object system built on macros, and a (controversial) LOOP macro that is its own DSL for expressing iterations, allowing idioms like (loop for i from 4 downto 1 collect (* i i)), which would return (16 9 4 1).

Macros can be very useful. They're pretty much essential because of how Lisp works, so every Lisper ends up writing a few, often to substantial benefit. That said, macros can also be abused to make write-only code. You have to be careful with them, and the maxim, "Never use a macro when a function will do", is certainly worth heeding.