What pointers basically are is not particularly hard to grasp. What is harder to grasp it what can be done with them and how you can shoot yourself in the foot with them in non-obvious ways.

I think I only understood much of it once I learned Rust, because you realize: Ah, that thing I once did in C is something that maybe ahouldn't be possible at all without extra steps. Even if I were to nwver use Rust again, this definitly helped to understand how to use pointers more safely.

I think that pointers are a tool to control a CPU's indirect addressing modes from a higher-level language.

Exposure to an assembler makes pointers easy to understand.

What's difficult to understand about pointers isn't the concept of a pointer itself, or even * and &, it's the fact that working with pointers requires you to simultaneously understand different abstraction levels. While it's not unique to pointers, and it's in fact the case for most nontrivial programming tasks, what's unique about C is that pointers are so pervasive you can't really do anything if you don't understand how to work with them.

IME languages like Python aren't any easier than C to work with (ignoring UB issues of course), but it's certainly the case that you can probably kinda sorta get your job done with Python even without understanding the first thing of what you're doing, and that's not happening if you write in C.

In C, pointers require you to think deeply about the ownership and lifetime of any "allocated object" at runtime. How long does it live, who is responsible for the deallocation, how many pointers does your program hold to that object (dangling issues). Ultimately, it can lead to a cleaner design if these issues are taken seriously up-front.

I don't disagree with that, but most cases fall within a pretty clear pattern:

- typedef struct { ... } foo

- foo *foo_create()

- void foo_destroy(foo *)

- a bunch of functions that take foo* as their first arg

which is kind of the same as a class and only more error-prone.

I say this as someone who actually _likes_ C, but the manual memory management model is very often unnecessary, confusing, repetitive. There was an idea some time ago of a language extension that would extend the concept of automatic storage duration to allow an explicit destructor to be called when the variable goes out of scope, like <close> variables in some languages. I genuinely think things like that would make the language a bit more ergonomic without fundamentally changing its nature.

That's why I tend to always prefer automatic and static storage to dynamic allocation wherever possible, especially in cases where you don't have "N" items or "N" cannot possibly exceed a certain small value. Also, allocation/deallocation of the certain object need not be defined within its module. It should be up to the caller to decide whether to allocate the object on the stack, statically or dynamically depending on the caller's situation:

    foo f;
    foo_init(&f);
    foo_destroy(&f);
    ...
    foo *g = malloc(sizeof(foo));
    foo_init(g);
    foo_destroy(g);
    free(g);

That is of course fine, with the drawback that it requires a public definition of the foo type.

I would write the allocation as

    foo * const g = malloc(sizeof *g);

to avoid repeating the type name and "lock" the allocation to the variable. If the type on the left hand side ever would change, this still does the right thing.

Interesting, I tend to prefer a create and destroy function that allocates and frees the structure. That way you can have foo without it being initialized, and you cant have foo been freed without being de-initialized. Where do see the value in being able to move it between different memory types?

In games and embedded systems, a very common pattern is batched heap allocation: a single allocation of an array of foo, and/or an array of foo mixed with other types, or sometimes a memory managed data structure like a memory pool. This is one big reason that the C++ Standard Template Library was shunned in game studios for a long time; it automatically did it’s own heap allocations, sometimes at inopportune times IIRC like in constructors, and didn’t let the caller choose. EA wrote their own version (EASTL) that was game friendly because it allowed control of the allocator.

In GPU programming, there are actual different memory sub-systems, with different sizes and different performance implications, so it’s critical that the caller is able to do the allocation & deallocation any way they want. This is why most well designed GPU APIs rarely allocate GPU memory inside the API, but instead are designed to work with caller-provided pointers to buffers.

Coming from the embedded world, I am far more used to the style bluetomcat showed where the caller handles allocation. It takes more work to use, and you have to make sure to not use it before initializing or after destroying, as you said. The advantage is that the caller has full control of memory allocation. The object can be a local variable, a static, malloc:ed, come from a custom allocator (say a slab allocator), or be a part of a larger struct.

That makes a lot of sense. I do some embedded development, but when I dont, I tend to prefer heap just because my memory debuggers are much more capable working with heap memory.

In the case you show, foo would have to be a struct that doesn't contain pointers to additional allocated memory, but its a entirely valid use case and pattern.

Calling malloc and free, has a cost associated with it, that the stack doesn't. But stack can in some cases, like with recursion be scary to use, because you dont know where it ends. If malloc returns NULL you know you have found the end and can do something reasonable.

Thanks for you insight!

I always lacked a good discussion on the subject of allocation/deallocation strategies. Is there some recommended good read?

It’s not standardized, but if you’re using gcc, you can use the cleanup attribute for this: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attribute...

IIRC clang implements it as well, but I wasn’t able to find a reference to it in their docs.

I’m actually implementing that right now for my own use, as a pre-processor.

There is a much more advanced design and implementation at “A defer mechanism for C” (December 2020): https://gustedt.wordpress.com/2020/12/14/a-defer-mechanism-f...

For my own purposes, I think I can live without handling stack unwinding so I continue working on my pre-processor.

Since the pre-processor is not yet finished, there I use a vector¹ of {.pointer, .destructor} where I put objects after initialization, with one macro at the end of each managed scope that calls the destructors for that scope in reverse order, then another macro meant for the function exit points that calls all the destructors in that vector. This has been built many times before by other people of course, it’s just an exercise to see which difficulties arise.

¹ Vector, growable array: I did my own trivial type-generic growable array, with the classic {.capacity, .len, .items}, but again there are many previous implementations. The two I’ve found more interesting are:

- “C Template Library (CTL)”: https://github.com/glouw/ctl

- “Klib: a Generic Library in C”: https://github.com/attractivechaos/klib/

For a vector alternative, I find this one easy to read and use :

https://github.com/tezc/sc/tree/master/array

It is just an array of your type, e.g int *numbers, so you have type info in debugger as well.

No type safety and broken alignment. No thanks.

Isn’t it type safe? e.g you can’t add char* to a double array.

What do you mean by broken alignment?

It lacks type safety because it's not possible to distinguish between an sc_array and a pointer so there's no way of detecting that someone passed a char * that nobody had ever called sc_array_create on to sc_array_add for example.

The alignment is broken because nothing in the C standard guarantees that the elems member of sc_array will be aligned correctly for any possible element type.

I also spotted another problem, in sc_array_init the code `void *p = a` is also not guaranteed to work. In an example snippet such as `int iv; sc_array_create(iv, 0);` expands to `sc_array_init(&iv, sizeof iv, 0)` so the type of the expression `&iv` is `int *` which is then being converted to `void *` in the function which is actually not allowed by the standard. This is also the reason why if you were writing a wrapper around realloc which exited if the allocation failed you would still have to pass in the current pointer with void * and return the new pointer with void *. This could be applied here actually as an easy fix but it indicates even further to me that the author of the library is taking a very leisurely approach to writing conforming C. This pattern also appears in the other two functions though and I'm not sure if in those cases it's something which can be easily fixed.

With a bit of trickery CTL works much like the STL, in that containers can be built of containers:

    #define P
    #define T int
    #include <vec.h>

    #define T vec_int
    #include <deq.h>

A deq_vec_int - analogous to std::deque<std::vector<int>> - is a neat example.

There's also a CTL fork that might be of interest: https://github.com/rurban/ctl

I also like the Ken Thompson extensions from which part of Go was inspired by

    struct X { int a; }
    struct Y { *X; int b; int c;}

    void add(Y* self, int number) {
      self->a += number;
    }

    Y y;
    y.a = 10; // composition
    y.add(1) // y.a = 11 now

This alone would simplify C coding so much without taking any power out of it.

The other extension i would add is some sort of interface or protocol.

As soon as you can do something like "y.add(1)", having a generic contract to refer to things without having to know its concrete type is some of the good things from the OOP world.

With this you would also be able to call some cleanup code and even a initializer.

This is still C and its still much simpler than C++, and yet almost as powerful.

C should propose these kind of things even if it was not that conservative and it would retain a lot of coders that migrate instead giving C barely evolved from its 70's roots.

Yeah, except when it doesn't follow the pattern: for example, there is no foo_destroy(), you're supposed to "just" call free() when you're done with it. Used to be very common (not sure how it is now), and very frustrating when you link against non-standard allocators.

> In C, pointers require you to think deeply about the ownership and lifetime of any "allocated object" at runtime.

Unless you decide you use libgc, presumably?

These are good points. I can sometimes feel that Python is more pointer-y (?) than people expect, with stuff like:

    a = {"one": 1, "two": 2 }
    b = a
    b["two"] = 99
    print(a["two"])

The above prints 99, since "b = a" does not copy the value (the dictionary) but just the reference to the value ("the pointer", kind of). This is surprising to some people.

I agree, which is why I'm glad the first language I learned was C. I don't get to write a lot of C at work, but the concepts the language teaches you are the very fundamentals of programming.

I know it's probably baseless, but I can't shake the feeling that people who learn modern languages before learning C are just making their own lives harder.

I think the same goes for learning assembly before learning C. A simple assembly-based computer [1] should be the first programming target in every "introduction to programming" class. After that, C becomes obvious and those who struggle with the fundamntals can be guided to more appropriate careers instead of dumbing down the tools everyone uses.

[0] https://www.instructables.com/CARDIAC-CARDboard-Illustrative...

This is one part (of many) of perl that I like, that it exposes the pointer-y parts to the programmer (to abuse :))

    # hash
    my %a = ( 'one' => 1, 'two' => 2 );
    my %b = %a;

    $b{ 'two' } = 99;

    # prints 2
    print $a{ 'two' }, "\n";

    # reference to hash
    my $a = { 'one' => 1, 'two' => 2 };
    my $b = $a;

    $b->{ 'two' } = 99;

    # prints 99
    print $a->{ 'two' }, "\n";

The real difference is Perl has typed variables, such as hashes or arrays, which other languages typically do not. Which does make it convenient when you need to clone the object, such as "%b = %a" example. Perl borrows the sigil concept from Bash and other shells (which makes sense, given the historical context of Perl). The only other language I know of that uses sigils in such manner is BASIC. Other languages, such as Ruby or Common Lisp, use them as syntactic convention rather than as a feature that the compiler/interpreter understands.[1]

Most dynamic languages expose the data as references. In fact, the one thing that trips up JavaScript developers (especially in React) is that they do not understand how references work. I see senior and lead developers inadvertently doing mutation all the time. Or getting incredibly paranoid that two identical strings, for example, do not equal each other in the strictest sense in JS. They also throw in memoization everywhere due to their fundamental lack of understanding.

You can always tell the developers that do not have C/C++/Pascal experience.

[1] https://en.wikipedia.org/wiki/Sigil_(computer_programming)

Reminds me of the Python trap that I fall into every few months because I forget about it:

def foo(mylist=[]): mylist.append("a") return mylist

mylist is only initialized once, so the function will actually return one more "a" with each function execution

It's a feature! Use this memoization pattern and you'll never forget it

  def foo(x, cache={}):
    if x in cache:
      return cache[x]
    val = cache[x] = x*x+1
    return val

But, be warned, there's no mechanism for cache eviction; use @lru_cache if you don't know ahead of time that x will take a reasonably small number of values

It's also similar in Java's pass-by-value vs pass-by-reference (pass as well as copy), if I recall correctly.

Actually, most languages only have pass-by-value, being “pedantic” (but it is actually a useful distinction). I’m sure there are other’s but out of the majorly used languages, only C++ has actual references. Pointers are passed by value, that is the pointer value gets copied. References are a language level construct.

Here is a great “rant” at it in case of Java:

http://www.javadude.com/articles/passbyvalue.htm

People that can’t comprehend arrays/lists in Python, especially multi-dimensional ones, would have had no chance in C

Pointers seem easy until you understand the pointers you're working with do not correspond with the basic hardware model you have in mind:

https://www.ralfj.de/blog/2018/07/24/pointers-and-bytes.html

https://www.ralfj.de/blog/2020/12/14/provenance.html

https://www.ralfj.de/blog/2019/07/14/uninit.html

Pointers aren't that complicated but C syntax is misleading imho until someone comes along and tells you that the declaration syntax "follows use", which does not seem like the greatest idea. Then you get used to it and forget about it but when you don't know the principle behind declaration syntax, it does not help you reason about the language.

Yup. I spent a ton of time trying to make sense of pointer syntax (* operator/modifier), until I realized that it didn't really make sense (except for the simplest cases) and then I could get on my life.

There's actually a logic to it, although not an intuitive one at all, and it's that declaration should follow use. That is `int *ptr;` is a pointer to an int and to get the int you have to use * on it.

The logic is that by declaring `int *ptr;` you effectively say that `*ptr` is an `int`. Only the first token (which must be a type) is syntactically special, the rest is normal expression syntax.

Same goes for arrays, `int *x[30]` just says that `*x[30]` is an `int` (well, technically it's out of bounds by 1), thereby we're declaring an array of 30 int-pointers.

Of course, that hasn't been entirely true since function declaration changed with ANSI C in the late 80s or so.

But int-pointer is the type of it. It makes no sense that it “sticks” to the variable name.

There's a type and an identifier. In `int *pa[10]`, the identifier is `pa` and the type is `int *[10]`. A declaration simply specifies a name (the identifier), some operations around it (pointer dereferencing, array subscription, function calling, grouping parentheses) and the terminal type that you get after applying all those operators in the precise order as specified by operator precedence rules.

When using that variable with the allowed operators, the resultant type of the expression is determined by the declaration:

    pa;     // int *[10]
    pa[x];  // int *
    *pa[x]; // int

Thank you for the concise write up, but I think lowly of the [] syntax as well.

And unary operators on an identifier are a different thing, though I similarly don’t think adding one at the left, another at the right is a good thing. But I have trouble with the declaration site as well.

It’s getting out of reach of my knowledge, but I believe C’s grammar being hacky is in part due to that as well.

I didn’t find exactly what I was looking for but here is some more on it: https://pdos.csail.mit.edu/archive/l/c/roskind.html

The reason the asterisk naturally "sticks to" the name is simply the possibility to write a series of declarations like this:

    int a, *b, (*c)(int);

Which is a questionable saving of 2 lines and a few characters in a language where huge repetitions are a given, due to lack of templates/generics whatever.

You missed the whole point. Yes, it makes no sense until you grok the logic of it and then it starts making sense (while still being unintuitive). Re-read the comments you are responding to.

Yeah I get that it has logic (though by being pedantic, anything we can write a parser for has logic, but I’m sure we could create some stupid languages), but as you yourself note, not very intuitive. And as far as I know, it doesn’t ease the work of parser-writers.

  int * a, b;

Here it sticks to a.

Then typedef comes along and ruins everything:

  typedef int *  pint;
  pint c, d;

Pointers are like swazzles[1]. The construction is very simple. The principle behind how they work is very simple. Learning how to use one well enough that you can (a) consistently make it do what you want, and (b) not injure yourself in the process, though, is no mean feat.

[1] https://www.atlasobscura.com/articles/swazzle-punch-and-judy

Well, you've demonstrated that memory addresses aren't that hard. But you've also demonstrated how easy it is to get undefined behavior in C programs.

C's pointers aren't memory addresses. Ok, they tend to be represented as such at run time, but that's not what the spec actually says the are. And as far as compiler authors are concerned, they can do anything they want as long as it's within spec. Further, the spec even requires some additional behaviors pure memory addresses aren't capable of. See https://www.ralfj.de/blog/2020/12/14/provenance.html for examples of the extra requirements.

Compared to that mess, lambdas are trivial. They're just functions.

> If you know how to walk down a street and stop at the right street number, then you have used pointers. And if you've ever observed that one tall building may "cover" a range of street numbers, such as 200-220

I see that as a European, I have virtually no chance to understand pointers using street numbers. :)

(Fortunately I've never had problems either with lambdas or with pointers.)

C is an abstraction over assembly, really (benefits being that it is simpler by being more abstract and portable across CPU types).

I've always thought that an introduction to CPUs (can take a simpler one as example) and how they work, how memory is (usually) organised, and to assembly would go a long way in helping understand many programming issues and C.

My experience is that C or programming concepts are often taught in a very abstract/mathematical way, which can be hard to grasp compared to a more practical approach.

If you take a concrete example where memory is effectively an array and indices are addresses (which holds true for most cases and, in any case is a good example) then understanding pointers becomes basically common sense and notations are simply conventions of the language you're using.

Treating C as an abstraction over assembly is a surefire way to step into all the thousands of sharp edges C has. In fact I would hazard a guess that the majority of bugs found in software written in C are a result of programmers treating it as a portable assembler instead of a language for programming an abstract machine. So many incorrect assumptions arise as a result of telling people to treat C as a portable assembler that I think it's safe to call it an extremely bad bit of advice.

I was discussing pointers. I only commented in passing about C being portable over assembler. You could rephrase this as it being a language for programming an abstract machine and it would not change anything about my comment (nor would it change the fact that C is an abstraction over assembly)

Thank you for your reply...

The fact that you didn't use the term "portable assembler" when saying: "If you take a concrete example where memory is effectively an array and indices are addresses (which holds true for most cases and, in any case is a good example)" doesn't change the fact that the statement makes links to how actual machine memory operates on what you personally think is a usual machine. This is really not a good idea because it encourages people to think of pointers as all existing in the same place and encourages erroneous thinking such as subtracting two pointers to different objects or confusion surrounding why the numbers you get when you `int a; char b; printf("%p, %p\n%p, %p\n", &a, &a+1, &b, &b+1);` differ in separation. Also subtler errors arise such as people assuming that all pointer types are effectively equal as long as you convert back to the correct pointer type when it comes to using the pointer (really subtle issues like people assuming that you can convert `int *` to `void *` safely).

The "very abstract" way C is taught actually prevents people from making such assumptions by not priming them to make them. The fact that people get complacent and start to lean on their understanding of (what they think are) real machines to write C is the result of the bugs I mentioned in the previous response.

Like my professor used to say, if you ever saw a pidgeon, you know what vectors are.

For anyone who wants to learn about pointers I can recommend studying a language simpler than C like for instance Oberon where pointers are more restricted. Having a look at Oberon can also broaden your view even if you know pointers in C.

  https://www.miasap.se/obnc/oberon-report.html
  http://people.inf.ethz.ch/wirth/Oberon/PIO.pdf

Actually, this could confuse things even more, because the C pointer and Oberon's POINTER TO have very little in common.

When pointers are used to build dynamic data structures the differences are merely syntactical.