What always amazes me: shouldn't the build system itself be as easy as possible to build (low requirements on the compiler, minimal dependencies, platform agnostic, etc.), e.g. just like "gcc -O2 buildsystem.c"? Also almost all of these systems seem to suffer from the same problems that were discussed (and solved) in the early years of software engineering, and e.g. hardly support modularization/encapsulation or static type checking. Cmake and meson are huge and complex, with a peculiar dynamic language each, not easy to install and use, and usually a factor ten bigger than what I want to build; also beast itself requires make, a recent gcc version and even flex and bison (so it doesn't e.g. run on windows, does it?).

How sqlite constructs the single source/header pair looks ideal. Various efforts have bundled lua and some libraries into a single C file too.

Check in that generated C file and you have trivial 'bootstrap' for everyone else. Ideally have some CI that checks the dev files still generate that exact C file on request.

Turns out I forked the project four years ago. Slightly different implementation in that I expanded the #includes to get a single file I can copy around and hacked in a few libraries (libuv, lfs and penlight iirc). Not totally pleased to discover that was so long ago.

Great to see that cc onelua.c gets the job done today.

We may want a more suitable tool that is first of all more user-friendly (like Beast), allows you flexibility in defining custom tasks (like Beast does).

Secondly, we simply may not have gcc to compile the all.c file (as mentioned in comments below). We just want a unified tool (an CLI preferably) that does all this for us. This is what Beast does. Make and Ninja are long standing build systems that prove my point.

Plus, you get Beast binary as a standalone executable. Check it out here: https://github.com/GauravDawra/Beast/releases/tag/v1.1.0

Second, you are listing tools like cmake and meson. Note that these are NOT build systems but they are META BUILD SYSTEMS. Meta build systems are tools that don't really build your project themselves... instead they rely on tools like Make, Ninja and Beast to build it for you. You must have seen that Cmake emits a Makefile (or Ninja file depending upon your choice). So comparison in speed to meson will not be appropriate. (Let us just call them high level build tools)

About your third query... Well Beast is easy enough to compile. If you don't want to compile it, you can directly download the system binaries at https://github.com/GauravDawra/Beast/releases/tag/v1.1.0

I'm currently using Make to compile Beast because I originally compiled it using Make. It will soon move on to CMake. Please note that it is totally OK to use flex and bison. You don't even need them since I have already provided the cpp/h files generated by them on github!

That's technically true, but you could just integrate the Ninja source code into such a meta build system and run it e.g. with a different command line option and then at the latest the attribute "meta" would be no longer relevant. "meta" in this context doesn't actually mean "on a meta level", but just that the tool is a transpiler from a high-level to a low-level build description, instead of a full execution engine. make (in contrast to Ninja) has features for both levels. Personally I would prefer an integrated system, so bootstrapping would not involve two (artificially) separate parts. But as I understand your message is that Beast is rather a Ninja alternative than a Cmake alternative. That's ok, although we already have Ninja and it claims to be a very fast build tool. So in the question we then just replace Cmake/Meson by Ninja.

> since I have already provided the cpp/h files generated by them on github!

Ok, I see. So it then also runs on Windows? How do you build the tool on Windows?

EDIT: actually if you're interested in GN, which has some very good concepts not seen elsewhere, I've written a tool to view and navigate GN source trees, see https://github.com/rochus-keller/GnTools. I usually use it with the Dart SDK source tree.

Moreover, the benchmarking I'm doing currently (with the Nimble release) shows that not only Beast matches Ninja's speeds, it overtakes it for different configurations and parallelism levels.

Right now it is not supported for Windows. If you have any contribution that can help in this direction they are more than welcome :)

I will surely check out your repository!

I don't know how you would write and deploy a build system when there is no implementation for the language it uses; this looks like an unsolvable bootstrap problem to me; you would need a precompiled build system to build the build system, and that on all platforms you want to build; and anyway I don't think that a build system language should be Turing complete (or otherwise the language should at least support modularization and static typing).

Bazel is certainly great for a lot of use cases, but it is one of those Google colossi where you have to hire a specialized team to manage and run your build. GN has a similar issue in that in practice there is an insurmountable dependency of a gazillion of Python scripts (depot tools and the like), even if GN and Ninja are written in C++ and only have standard dependencies.

Anecdotally I've worked on projects that have thousands of scripts that could be built with a 5 line make file, and other projects that have a few tens of files that needed a 200 line Webpack config. I don't think build complexity is a function of project size.

It even seems to copy make's syntax, which IMO is a missed opportunity. One of the issues with Make is that the syntax is clever until you need to invoke multiple lines of shell script, like a loop. Then the tab-based cleverness becomes a pain.

Plus, it is an order of a magnitude faster than Make. All in all, I don't agree with the fact that it's syntax is similar to Make. Well for one thing it has legible special variable names like 'out' and 'dep'.

So yes, while it is a low level build system, it is an improvement over Make in terms of syntax and speed.

Make is rarely a big bottleneck, except in very pathologically written projects, and when doing incremental builds. I think the main problem with big make setups is recursive make, which prevents it from having a global view on the project, and forces the main make to execute many sub-makes recursively, all of which has overhead.

Eg, if you have src/lib, and src/bin, and the src/bin project depends on the src/lib library, then in self-contained, recursively invoked makefiles (or anything else), you won't get a correct behavior when a header in the library changes.

I recently wrote some retrospective notes about build system design in this area that seem like they might be useful for someone in exactly your place:

https://neugierig.org/software/blog/2022/03/n2.html

Great work!!

* Task runners (rake, simple Make rules, shell scripts) etc. Basically convenience scripts for devs. * Hermetic build systems

Hermetic build systems (Bazel, Nix package manager) can be 100% confident that there aren't any secret implicit dependencies and this unlocks some really great stuff such as:

* Complete confidence in incremental builds at all time. It isn't possible for something secret to have changed * Excellent build caching, because you know all the inputs match the cache

The classic example of this is C or C++. The build step is `gcc -c main.c -o main.o`, but that build command depends on a load of header files included by main.c. Both system ones and ones local to your project. If you change a header and rerun your build rule, it only knows about main.c, knows that it hasn't changed (by hash or timestamp) and doesn't do anything. You end up having to rebuild a load of stuff just in case. Your usual system (e.g. make) has no idea about this. Various solutions exist - Beast talks about the 'get gcc to tell you about the dependencies' approach but now you've got another file to generate and manage and have in your build system, and you still can't be sure it's right.

There is a (good) argument to be made that these are problems that are only unmanageable in big software projects, and the rest of us should just take advantage of the simplicity of make (or newer better solutions like Beast). This is a similar argument to the one being made in another post about K8s and 'early optimisation'. But this is why every time someone posts a new build tool, I always hope it's going to be about 'power and reliability of Bazel' but as friendly to get going with a basic make rule.

Wanting to know more, I found this:

"Hermeticity: This page covers hermeticity, the benefits of using hermetic builds, and strategies for identifying non-hermetic behavior in your builds." https://docs.bazel.build/versions/main/hermeticity.html

Sounds great.

Ages ago, my teams had a policy of "one button build". Install VS C++ on a new box, open the project (from source repo), hit "Build". Tada.

We could rebuild any revision on demand. Terrific for reproducing regressions and delta-debugging.

In the Java world, with (misuse of) maven, gradle, jenkins, etc. attaining reproducible one button builds is quixotic.

For hermetic builds, everything would be digitally signed (SHA256), right? There's a spec for signing Linux kernels, which I can't quickly refind. But the idea is to apply that strategy to everything, right?

That sounds perfect.

For caches you just say 'here's the SHA1 of the inputs and the cache server can just give you the output.

Basel does this for each build command, e.g. for each object in a big C program. Nix does it for each package.

Have you seen https://please.build? It is very close to what I think the answer to "What if Blaze was made today?" is. It is conceptually much simpler and doesn't suffer from some of the things that need to happen in Google-scale projects/teams. For example, they haven't done the same gross thing with proto_library that google did. You only define one and it only builds the bindings that are actually needed by the targets you are building.

Doesn't support Windows which is a bit of a shame, but maybe manageable. Will have to check it out properly at work tomorrow.

You should make your value proposition clear in your project README, especially if your syntax and overall model appear to be quite similar to an existing tool. Saying you focus on “ease of usability and speed” doesn’t tell me much.

Make does not cope with this at all. Yes I'm aware of all the workarounds and they all fall over at some point. Make works perfectly well on Windows for instance, except for the whole generation of Windows that had the users under "Documents and settings". Was almost always impossible to build code in the user home directory.

or it can be for any language, just put the build command line in beast file? thanks.

You are right in saying that since any command can be put in the beast file, it is suitable for building a project in any language.

It is a general purpose build system, with focus on speed and usability!

- examples for using it for android or iOS, or java project, and some numbers showing its better performance?

- how to use it for an android project? gradle handles it all with target(sub targets tree structure of the build tasks...), if use Beast, will still be using gradle? or the build tools gradles is calling for different things (compile, packaging,signing...)

2. You can use it for any project you want if you know what are components of your project depend upon what other components. So as I see it, gradle is a build system for Java project in itself. Beast will not be using gradle if you are trying to compile a Java project. BUT.... you can for sure call `gradle` command from inside the beast.build file. Wherever build.gradle file is stored. So that in this case, beast provides you with an easier to use build system. Let us see an example:

In your beast file, you can have something like:

  build hello:
     ! gradle -q hello

  task hello {
    doLast {
      println 'tutorialspoint'
    }
  }

  build pre:
     ! echo 'pre script'

  build proj:
     ! gradle hello

  build pre:
     ! echo 'pre script'

  build proj: pre
     ! gradle hello

1. https://taskfile.dev/#/

It can do anything you want if you can provide it with proper shell commands and define targets. This can not only be used for tasks but also for physical building targets that can be built on the machine.

I believe that the project you linked here is itself also a build system and not just a task runner. But again, it uses yaml, which might not be best while defining custom things related to a build system

CMake and Meson on the other hand are Meta build systems. They do not run any build commands on their own. They use other build systems like Make, Ninja to carry out everything. Think of them as a fancy wrapper around build systems. Beast is here as an alternative to Make and Ninja.