News for hackers. Stuff that matters.
Re:How safe is Zig? by AndyKelley on Thursday June 23, 2022 @03:19PM (#31853964)
I have one trick up my sleeve for memory safety of locals. I'm looking forward to experimenting with it during an upcoming release cycle of Zig. However, this release cycle (0.10.0) is all about polishing the self-hosted compiler and shipping it. I'll be sure to make a blog post about it exploring the tradeoffs - it won't be a silver bullet - and I'm sure it will be a lively discussion. The idea is (1) escape analysis and (2) in safe builds, secretly heap-allocate possibly-escaped locals with a hardened allocator and then free the locals at the end of their declared scope.
Re:How safe is Zig? by nine_k on Thursday June 23, 2022 @11:23PM (#31857448)
I would rather prefer the compiler to tell me: "Hey, this stack-allocated variable is escaping the function's scope, I can't do that! Allocate it somewhere outside the stack."

Maybe the compiler could offer me a simple way to fix the declaration somehow. But being explicit and transparent here feels important to me; if I wanted to second-guess the compiler and meditate over disassembly, I could pick C++.

Re:How safe is Zig? by flohofwoe on Friday June 24, 2022 @02:19AM (#31858343)
Since HN upvotes are invisible in the UI: +1. IMHO escaped locals should be an error, but not a hidden allocation.
Re:How safe is Zig? by randyrand on Thursday June 23, 2022 @07:39PM (#31856412)
I feel like the most user friendly solution for Use After Free is to just use Reference Counting. Basically, just copy Objective-C's "weak_ptr" design.

For every allocation, also store on the heap a "reference object", that keeps track of this new reference.

  struct reference_t {
   // the ptr returned by malloc
   void* ptr;
   // the stack/heap location that ptr was written to.
   // i.e  the reference location.
   void* referenceAddr; 
  }
Reference track: every time ptr is copied or overwritten, create and delete reference objects as needed.

If ptr is freed, visit every reference and call *referenceAddr = NULL, turning all of these references into NULL pointers.

Re:How safe is Zig? by kaba0 on Friday June 24, 2022 @06:25AM (#31859813)
Reference counting is an incomplete solution it itself - it leaks circular references, and on multithreaded programs it is quite slow. Though some optimizations could be done statically to improve on that.

All in all, a mark-and-sweep will likely be faster (in throughput at least) let alone a good GC.

Re:How safe is Zig? by randyrand on Friday June 24, 2022 @01:06PM (#31865796)
Circular references wouldn’t matter.

The programmer would still decide when to free the object, not some automatic system. Manual memory management, with Ref Counting just to add some additional Use After Free checks.

Re:How safe is Zig? by jhgb on Friday June 24, 2022 @03:33AM (#31858741)
Wouldn't this, if applied by default, complicate Zig's easy C interoperability? I don't quite see how plain C code could play well with this.
Re:How safe is Zig? by skullt on Thursday June 23, 2022 @04:26PM (#31854709)
Does that not contradict the Zig principle of no hidden allocations?
Re:How safe is Zig? by kristoff_it on Thursday June 23, 2022 @04:45PM (#31854918)
I don't know the precise details of what Andrew has in mind but the compiler can know how much memory is required for this kind of operation at compile time. This is different from normal heap allocation where you only know how much memory is needed at the last minute.

At least in simple cases, this means that the memory for escaped variables could be allocated all at once at the beginning of the program not too differently to how the program allocates memory for the stack.

Re:How safe is Zig? by messe on Thursday June 23, 2022 @06:32PM (#31855907)
Static allocation at the beginning of the program like that can only work for single threaded programs with non-recursive functions though, right?

I’d hazard a guess that the implementation will rely on use-after-free faulting, meaning that the use of any escaped variable will fault rather than corrupting the stack.

Re:How safe is Zig? by wavesquid on Thursday June 23, 2022 @10:57PM (#31857323)
No need to limit to single-threaded: as long as we reserve enough space in the TLS area it's possible to work with multiple threads.

Zig has future plans to require recursive functions to declare their maximum stack memory usage up-front, so that will provide the rest.

Re:How safe is Zig? by adwn on Thursday June 23, 2022 @11:56PM (#31857657)
No need for recursion or multi-threading: If you call the function in a loop and don't release the escaped local unter after the loop, and if the number of loop iterations isn't statically known, then it's impossible to pre-allocate heap storage for that local variable.
Re:How safe is Zig? by remexre on Thursday June 23, 2022 @06:24PM (#31855837)
Could this be integrated into the LLVM SafeStack pass? (I don't know how related Zig still is to LLVM, or if your thing would be implemented there.)
Re:How safe is Zig? by randyrand on Friday June 24, 2022 @12:49AM (#31857917)
Just another idea for use after free:

What If we combined the 'non-repeating' malloc idea with 128-bit uuids?

malloc would just return a 128-bit uuid, and to get to the data ptr you'd need to consult a hash table.

  dataPtrArr[hash(uuid)].dataPtr = dataPtr
We'd check if it's been freed by checking:

  dataPtrArr[hash(uuid)].uuid == uuid
Re:How safe is Zig? by flohofwoe on Friday June 24, 2022 @02:15AM (#31858323)
At that point it's better to use 'tagged index handles', but IMHO that's outside the scope of the language (but maybe an option for the stdlib):

https://floooh.github.io/2018/06/17/handles-vs-pointers.html

(but for this an "auto-decaying pointer" would be nice which cannot be stored outside the stack and cannot be "carried across" function calls.

Re:How safe is Zig? by randyrand on Friday June 24, 2022 @05:14AM (#31859395)
Nice article. Similar idea ofc. Yes, that design could work tho all objects per pool must be the same size. Not sure how that would be better.
Re:How safe is Zig? by pjmlp on Friday June 24, 2022 @01:28AM (#31858098)
At which point is much simpler to introduce automatic memory management in some form.

That solution is basically how platforms like Psion or Symbian used handles due to memory constraints.

Re:How safe is Zig? by randyrand on Friday June 24, 2022 @01:44AM (#31858191)
IMO this is still simpler than automatic memory management, and the runtime costs are mostly fixed and predictable.

You also don't need to worry about ref cycles or GC pauses.

Re:How safe is Zig? by pjmlp on Friday June 24, 2022 @02:01AM (#31858268)
Assuming the mythical existence of the perfect developer that nevers does mistakes which handle to use, which certainly wasn't the case on Psion or Symbian projects.