Also where it gets worse and more expensive up to the point where people really can't pay for it anymore.

If you think about it, it should be possible to sell drinking water for 10$ per glass since people will buy it at that price. Just need to consolidate all of the supply by selling it on subsidized prices until all the competitors are bankrupt.

I'm sure they're excited by your suggestion though tempered by the fact that we did make it illegal for them to cut off your supply of drinking water if you're in debt. Not electricity, you can freeze or sit in the dark, but we decided that leaving the impoverished thirsty was a step too far and people might realise Capitalism wasn't such a great ideal after all...

This isn't really true of Goose, but it's avoiding 'the heap' in the C sense, not in the OS allocated pages of memory sense. One is a programming abstraction provided by the other, but GP is only talking about the programming abstraction. I have never encountered a program that couldn't have memory be explicitly tied to lexical scope, but it can result in less efficient use of it in some cases.

Goose in particular excites me because I have done similar design work over the years.

Also, I took a closer look at the actual benchmark results [2]. For memory usage, at least, the median and mode would show Goose using slightly more memory than the well optimized C++ and Rust. It looks like the mean is skewed in Goose's favor by a couple of benchmarks where it does extremely well due to its variable-length enums.

[1]: https://aardappel.github.io/goose/docs/tutorial.html#18-limi...

[2]: https://aardappel.github.io/goose/bench/results.html

Where are you seeing that?

Goose's stacks have quite some complexity themselves. They preallocate a ton of memory. The compiler has to magically match your code to each stack. You are not allowed to have nested growing types (e.g. creating an adjacency list is not so easy). And after all for some usecases you still end up using an allocator (Goose's linked list example ends up building a pool allocator for the linked list nodes!)

you will only learn about the thin or heavy wrapper that C is responsible for, the real magic is happening in the kernel and the device firmware

My toy stop/copy GC and allocation routines just have this:

    %define HEAP_SIZE (4 * 1024 * 1024 * 1024)

    .section bss
    ;;
    ;; Two heaps so we can flip-flop between them
    ;;
    heap_a:
        resb HEAP_SIZE
    heap_a_end:
    heap_b:
        resb HEAP_SIZE
    heap_b_end:

Nobody in their right mind (except musl) use brk(2) nowadays; mmap(2) is pretty much universally accessible, and brk() has been non-portable for more than 20 years already (POSIX ditched it in 2001) anyhow. For goodness sake, stop keeping this stupid API from VM-lacking PDP-7 alive, and let it go away!

See the NOTES section in man brk(2):

> C library/kernel differences

> The return value described above for brk() is the behavior provided by the glibc wrapper function for the Linux brk() system call. (On most other implementations, the return value from brk() is the same; this return value was also specified in SUSv2.) However, the actual Linux system call returns the new program break on success. On failure, the system call returns the current break. The glibc wrapper function does some work (i.e., checks whether the new break is less than addr) to provide the 0 and -1 return values described above.

The usual division of labour in most operating systems is the kernel manages the virtual memory and physical pages and provides a mechanism for coarse page-aligned allocation of virtual address for userland. These coarse mappings userland (malloc or something else) is responsible for carving up, and in that is much design space. Just look at mimalloc for example.

The kernel's own page allocation is often very simple (many get by with a simple freelist, some use buddies.) The allocators for the kernel's own datastructures are usually relatively simple at least conceptually. Bonwick explains the slab allocator - the almost universally acknowledged queen of all kernel memory allocators and mother of those of Linux, BSD, and beyond - in a 12-page paper.

As for the firmware there's basically nothing of an ongoing role for it after the kernel is in charge. Its role is limited for the most part to providing the kernel with a map of the physical addres space.

By reading malloc() you will not learn everything about allocation - that is a problem that spans multiple layers - but you will certainly learn a lot about it. To call it a mere wrapper is not reasonable.

   One of  my pet  peeves is  when people,  trying to  make a
   point, use code  that doesn't compile.  I  mean, it's fine
   when you're sketching  out code on a white  board.  But on
   an  article you've  had  time to  prepare,  it's a  little
   annoying.

   Take for instance, this code fragment:

   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..  }

   I'm pretty sure what they //meant// to write was:

   .. #include <stdlib.h>
   ..
   .. static void *ptrs[1000];
   .. 
   .. int main(void) {
   ..     for (int i = 0; i < 1000; i++)
   ..         ptrs[i] = malloc(16);
   ..     }
   ..     return( 0 );
   ..  }

   I  understand why  the  author produced  the  code on  top
   instead of the code at the bottom.  The code on the bottom
   includes  features extraneous  to  the conversation.   The
   include  directive  and  return statement  only  serve  to
   distract from the point the author is making.

   If you read far enough down,  you'll see he has links to a
   broken GitHub  repo presumably  with code  fragments.  (or
   maybe  GitHub itself  is  broken today  or  it decided  it
   didn't  like me  trying to  access the  repo from  an AT&T
   fiber line.)

   I'm sure when GitHub decides to behave and serve bits like
   it's supposed to, there will be compilable code so readers
   can understand  the author's intent within  the context it
   was intended to be viewed.

   Which is to say:

   1. It's probably  okay to have only code  fragments in the
      main part  of your  discussion.  Please provide  a note
      that  there is  code which  shows your  snippet in  the
      context  of a  compilable program.   And put  that note
      somewhere near the code fragment.

      Or... consider putting  a note at the  beginning of the
      article saying  something like "Compilable  versions of
      code  fragments  in this  article  are  available at  a
      GitHub  repository  referenced  at the  bottom  of  the
      page."

   2. Do  not depend on  GitHub to hold your  code fragments.
      You do not  know when it will be down  or which readers
      it will randomly decide to block.

   3.  I really  do  think  you should  include  more of  the
      "layout"  code.   When   reading  "The  Address  Space"
      section,   it  was   unclear  where   the  initialized,
      uninitialized and local came from.  I initially thought
      the  author was  using a  version of  C I  had not  yet
      encountered that defined these symbols.

   4.  Since  the code  uses  ``sbrk()``,  the author  should
      probably have  cautioned readers  the code  isn't POSIX
      compliant, but should compile  and run on Linux systems
      using gcc.   There's a  comment about  demonstrating it
      with  glibc, but  I think  that's not  a strong  enough
      warning since you can  compile programs with clang that
      are intended to link against glibc.
      
      I doubt there  are many QNX users out  there who expect
      random code from  the internet to run  on their systems
      without alteration, but it  seems good form to describe
      the environment you are targeting.

      Now I'm wondering if this is expected to work on Darwin
      as well as Linux.

   5.  It  might  be  a  decent  idea  to  order  the  SHOW()
      statements in  the code in "The  Address Space" section
      to follow  the order  of the diagram  at the  bottom of
      that section:

      .. SHOW( "local (stack)", &local );
      .. SHOW( "printf (libc)", printf );
      .. SHOW( "program break", sbrk(0) );
      .. SHOW( "malloc(16) (heap)", malloc(16) );
      .. SHOW( "uninitialised (bss)", &uninitialised);
      .. SHOW( "initialised (data)", &initialised );
      .. SHOW( "main(text)", main );

   6.  In  the  "Chunks  and  Headers"  section,  it  may  be
      interesting to mention what the flags in the allocation
      headers actually mean.

   7. glibc does late fill,  right?  I can't remember.  If it
      does,  it might  be  useful to  mention  in the  Demand
      Paging section since it affects the program logic.
   
   Now that I've yelled at the  cloud a bit, allow me to say,
   I enjoyed the article.  Things I liked:

   1. The author  used the exact strace command  they used to
      produce  the  output  discussed  in  the  text  of  the
      article.

   2. The graphics are quite nice.

   3. Despite grousing about  fragments being fragments, they
      do a good job of illustrating points made in the text.

   EDIT: After complaining about  GitHub not being available,
   it's finally come back.  The web interface is still borked
   for me, but  I can clone the repo mentioned  at the bottom
   of the page and view the local copy of the source.

Thanks, I'll keep that in mind, but don't get your hopes up

What I'd love is for an article that goes a step back in allocator theory instead of "forward" to jemalloc/mimalloc. Optimal resource allocation has a rich history in pre-computer fields whose general theories are applicable today. Replace "linear memory" with "paper roll" and "types" with "items", and instead of malloc you have the cutting stock problem! https://en.wikipedia.org/wiki/Cutting_stock_problem

The reason why allocator fundamentals are important is because there is no universally optimal solution. jemalloc "emphasizes fragmentation avoidance and scalable concurrency support" while mimalloc focuses on a bounded worst-case allocation latency. Meanwhile if you're writing extremely memory constrained code for certain embedded systems, you may still be best off using a simple bump allocator with no free! https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98... The optimal resource allocation strategy is always use case dependent.

I would argue that the missile performs the ultimate in physical garbage creation at the end of its flight.

I agree.

> This is mostly describing a kind of obsolete allocator

I disagree. glibc malloc is still extremely relevant and sees a lot of development, and uses exactly this strategy (unless an allocation request is too big, in which case it mmaps, just like the allocator described in the article).

Vudo malloc tricks by Michel "MaXX" Kaempf (2001)

https://phrack.org/issues/57/vudo-malloc-tricks

It has no way to specify custom alignment. So, if more than 8 or 16-bytes alignment is needed, custom user-code tricks are necessary (allocate more than necessary, round up).

It has no way to report how many bytes are actually allocated. Sometimes it may be useful, like if I reserve 9 bytes for a string and underlying allocation granularity is 16 bytes, I want to know that I can increase string size up to 16 bytes without calling malloc again.

It's a single entry point allowing no allocation size customizations. So, if I allocate objects of fixed size, I still need to pass a dynamic size parameter to malloc and it needs to perform branching in case if it does have some optimizations for particular sizes.

free has no size parameter, which is a mistake. This forces the allocator to maintain size values somewhere inside, which is often an unnecessary overhead, since user code already knows size (like for fixed-size allocations or containers storing explicit capacity).

With interface fixes described above a malloc implementation can be much more performant and (sometimes) even easier to use.