News for hackers. Stuff that matters.
Re:Why is my x64 process getting heap address above 4GB on W... by copx on Saturday July 11, 2015 @07:40PM (#9871984)
Pointer packing is popular with people wanting to write efficient interpreters for programming languages. The memory use and performance gains are massive and it was perfectly safe during the 32-bit era.

LuaJIT does it for example. Interestingly enough some programmers do this without even understanding how memory allocation works. For example the author of the newish wren language: https://github.com/munificent/wren

Somehow most programmers - including almost everyone who frequents reddit.com/r/programming - believe that "virtual address space" means that the addresses returned by malloc() will restart from (somewhere close to) 0x0 for every new process i.e. that it is safe to truncate pointers if you never intent to allocate more than ~2GB per process anyway.

Re:Why is my x64 process getting heap address above 4GB on W... by beagle3 on Sunday July 12, 2015 @06:59AM (#9873111)
LuaJIT does something weirder. It does double packing - using 48 bits (IIRC) of value inside a "NaN" double encoding. It is going to be a while before this becomes a problem, even on 64-bit systems - you need more than 128TB addressable space before this becomes a problem (and I suspect Pall will switch to a 51-bit (1024TB=1PB) pointer when that future is on the horizon).
Re:Why is my x64 process getting heap address above 4GB on W... by copx on Sunday July 12, 2015 @08:17AM (#9873246)
No that is wrong. Currently the x64 port of LuaJIT is limited to about 1GB [1] in the best case and can fail to allocate any Lua objects at all if the lower region of the address space is already completely consumed.

[1] http://lua-users.org/lists/lua-l/2010-11/msg00241.html

Re:Why is my x64 process getting heap address above 4GB on W... by beagle3 on Sunday July 12, 2015 @08:37AM (#9873286)
Thanks. A slightly later discussion I found is here[0]. However, this is not a limit imposed by the tag structure used by LuaJIT, but rather the Lua/LuaJIT GC (from a quick reading of the mailing list discussion - I might be wrong here). Apparently, there's a GC in the works that is supposed to resolve this, but it isn't there yet.

[0] http://lua-users.org/lists/lua-l/2012-04/msg00729.html