> a pinch of m4

nononononononononono for the love of everything please no

m4 isn't even a good esolang!

At the dawn of the age of PHP, I created a user management system (registration, verification, admin interface, …) that was based on well-established ideas (how login worked at Yahoo, Amazon, and every other process major site) but got no traction at all as a open source project. In any language that wasn’t PHP it would be necessary to write an “authentication module” which as about 50lines of cookie handling code. Multiple times I managed to out several existing apps together and make an advanced web site.

About 10 years ago the idea suddenly got traction once it was legitimized by the SAAS fad, I would tell people “don’t you know they’re going to end the free tier or go out of business or both?” and sure enough they did.

Anyhow, I bring it up because the system used M4 to interpolate variables into PHP, other languages, shell scripts, SQL scripts, etc.

Ugh, I know exactly how this feels. You resist the urge so hard to say “I told you so” and instead relish in the fact that you saw it. “The Way”, so to speak.

I remember having to write cgi cookie handling code. I remember having to write session-cookie sync code. PHP was a small slice of heaven in the cgi world. Until it wasn’t. Still, being able to import libraries of script functions without having to recompile was wizardry. The problem with php now is they let a certain product somewhat dictate their direction. Class namespaces with slashes is the ugliest design choice.

What was your oss project that couldn’t get traction?

It was called Tapir User Manager but the web site was down for some time. It was an open source failure but a career success because I used it around 8 projects including the arXiv preprint service, a voice chat service that got 400,000+ users, and the web site for our county green party (which had national impact.)

You need to combine it with Perl and a collection of other special passes, of course: https://web.archive.org/web/20180309134414/http://thewml.org...

Instead of `m4` or `sed` find and replace, the author should try `envsubst`. It's a program that replaces bash style variable references (for example `$TITLE`) with their value in the environment.

    export CURRENT="..."
    cat page.html | envsubt

The problem is that the $SOMETHING syntax is just too common if your site is a technical one, and you'll end up substituting too much.

You can specify which variable names are valid, reducing the likelihood of a collision.

I agree that `envsubst` is a good choice for this. Unfortunately, it is not part of posix, so you can't rely on it being present everywhere. But as part of gettext, it is still very common.

There is no such thing as a "pinch of m4". You start a clean project promising that you won't touch m4 this time. Then you add a small m4 invocation to save yourself from some boilerplate.

A year later, when you are trying to figure out why all instances of the word "cat" are silently disapearing from your website, you dig through 5 layers of macro expansions to discover that a junior dev tried implementing a for loop instead of copying it from the manual and messed up the quotation marks.

Having solved the immediate issue, you decide that debbuging your DSL is too hard, so you import M4 macro file you have been copying between projects. You then spend a day replacinf all usages of 'define' with your macro-creating-macro that adds comments to the output enabling your stacktrace generation script to work.

Next project, I am putting down a hard rule: no m4! (Except for maybe that one instance)

Please write more to this story

Not "this" story. Everything above has happened on several projects. The cat thing comes up because it is tricky to expand two macros next to each other without whitespace. So if you do:

  define(`foo',`hello')
  define(`bar',`world')
  foo bar
  foobar

You will get:

  hello world
  foobar

Working around this gets tricky, so someone inevitably ends up writing a cat-like macro such that you can do

  cat(foo,bar)

To get

  helloworld.

A side effect of this is that now "cat" is really "cat()" which expands to "". You can work around this by doing `cat'. However, if `cat' is used as an argument to another macro (such as a for loop), the quotation only prevents escaping the first time. When the for macro is expanded, the quotation marks are stripped, giving you just "cat", which gets expanded again. A correctly written for macro would add new quotes as needed, but I have never seen someone correctly write such a macro without just copying it.

Not sure if I have seen this interaction specifically with for and cat, but I have seen an interaction like it on almost every project that used m4.

You can place an empty "expansion" in that line to get the behavior you want without an additional cat-like function

    foo`'bar

I only know of this feature because recently read the manual page for m4, and it's mentioned rather early in there, but might have been not as well emphasized in past iterations of the manual.

For completions sake, though, the easy way to do it is:

    foo`'bar

The empty quotes make foo and bar separate words.

I've only ever used m4 via autoconf and sendmail configuration files, so I don't know if it's m4 that has the bizarre syntax or whether it's autoconf's and sendmail's use of it. I'm not sure I've ever tried to use m4 directly for anything.

That's why I wrote a "pinch" in the OC. m4 is arguably better than cat as a barebones template engine. The moment you start doing anything beyond simple includes and variable extrapolations, it is time to switch something more modern and robust.

I guess autoconf/sendmail still use m4 because there wasn't anything better at the time that doesn't come with a kitchensink attached.

I know that story too well. Finally, I thought that if I have to code, I should just use a programming language.

Now, I use nodeJS to replace every m4 file with mustache.js and some JS logic and I don't feel limited anymore. The complexity doesn't increase much.

I did that once. Never again.

Just because it worked for sendmail is not sufficient justification for anything.

A lot of older unix software config is complicated and cryptic.

sendmail, bind, apache, older X11, sudo are examples that come to mind.

no shit. not everything is yaml and emoji status lines.

but that's got nothing to do my mistake of using M4 when better tools existed.

Rather than relying on generic text substitution using m4 or perl or whatever, I suggest using SGML, the basis and common superset of HTML and XML, which comes with easy type checked text macro (entity) expansion for free or even type-aware parametric macro expansion. Where "type" refers to the regular content type of a markup element (ie. its allowed child elements and their expected order) but also considers expansion and escaping into attributes or other context such CDATA or RCDATA. Only SGML can extend to the case of properly expanding/escaping potentially malicious user comments with custom rules such as eg. allowing span-level markup but disallowing script elements, does markdown or other Wiki syntax expansion into HTML, can import external/syndicated HTML content, produce RSS and outlines for navigation, etc. Works well for nontrivial static site preparation tasks on the command-line; cf. linked tutorial and command line reference.

[1]: https://sgmljs.net/docs/producing-html-tutorial/producing-ht...

[2]: https://sgmljs.net/docs/sgmlproc-manual.html

What is sgmljs? There doesn’t seem to be any explanation on the site.

A comprehensive package for processing, converting, and serving SGML on the command line, on the server side, or in the browser; see [1]. Also features SGML DTDs (grammars) for W3C HTML 5, 5.1, 5.2, and Review Drafts January 2020 and 2023, which are the latest non-volatile W3C and WHATWG HTML recommendations/spec versions.

[1]: https://www.npmjs.com/package/sgml

I too had a small web site with M4 around 1999/2000. Why M4? Because I'd learned enough of it to be useful/dangerous when wrestling with Sendmail, and it seemed to do the trick (at least when the trick was simply "be easier than manually editing lots of HTML files every time there's a site-wide change").

I suspect I was never doing anything complicated enough to encounter the gotchas mentioned by other commenters...

haha, utterson also uses m4 for templating: https://github.com/stef/utterson/tree/master