Never create Ruby strings longer than 23 characters

Re:Never create Ruby strings longer than 23 characters by adriand on Wednesday January 04, 2012 @02:33PM (#3425505)

This was interesting and caused me to go off on a neat little tangent too.

I was curious about the VALUE declaration in:

    struct RString {
      long len;
      char *ptr;
      VALUE shared;
    };

From the Hacker Guide referenced, I found this definition:

    typedef unsigned long VALUE;

This is then casted, when needed, to a pointer to whatever type of struct you are dealing with. How does that work?

Well, on a 32-bit machine, if I'm correct in my reading, an unsigned long is 4 bytes in size and can contain these numbers: 0 to 4294967295

That last number is 4 gigabytes, which is the size of byte-addressable memory on a 32-bit machine. So VALUE can point anywhere. Neat!

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by charliesome on Wednesday January 04, 2012 @04:11PM (#3425918)

Fun fact: a VALUE is not only a pointer. Ruby takes advantage of the fact that pointers are aligned to a certain boundary (so a few of the least significant bits are always zero) and stores flags in the least significant bits.

For example, if the LSB is 1, then the VALUE isn't a pointer - it's a Fixnum. This is why the maximum size of a Ruby Fixnum is one bit less than the pointer size on the machine.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by jabkobob on Thursday January 05, 2012 @03:25AM (#3427951)

The Objective C runtime on MacOS 10.7 also does this for NSNumber objects.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by derleth on Wednesday January 04, 2012 @05:36PM (#3426257)

This is called 'type tagging' or just 'tagging'; needless to say, Lisp and Smalltalk and other interpreted language implementations have been doing it for decades now, and at one time there was direct hardware support for it in some architectures.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by dchest on Wednesday January 04, 2012 @06:07PM (#3426429)

Guile manual has nice details about data representation:

http://www.gnu.org/software/guile/manual/html_node/Data-Repr...

Tagging: http://www.gnu.org/software/guile/manual/html_node/Faster-In...

Another interesting technique is NaN-tagging (used in LuaJIT): http://wingolog.org/archives/2011/05/18/value-representation...

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by charliesome on Wednesday January 04, 2012 @05:41PM (#3426287)

Right! The 68k supported type tagging by having 24 bit addresses on a 32 bit machine. I'm not sure if allowing tagging was intentional when Motorola designed the 68k, but Mac OS did store flags in the upper bits of a pointer at some stage.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by vidarh on Wednesday January 04, 2012 @05:57PM (#3426375)

It was a quirk due to the number of address lines on the low end models, and it was a massive code smell, as 68020 and up could use 32 bit addresses so any code that did this would fail on machines with the faster CPUs.

Amiga Basic for example, wouldn't run on 68020 up because Microsoft used the upper 8 bits (amongst a whole slew of other horrible bugs and performance problems - it's probably the worst Microsoft product ever in terms of code quality)

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by derleth on Wednesday January 04, 2012 @05:55PM (#3426368)

> The 68k supported type tagging by having 24 bit addresses on a 32 bit machine.

Not quite: Having hardware support for type tagging means the machine knows that some bits are reserved for the tag and will therefore do arithmetic in a tag-preserving fashion. The 24-bit pointers in early m68k chips was just saving 8 bits worth of address lines.

The SPARC had tag support in hardware, in the form of special arithmetic opcodes. Further explanation:

http://compilers.iecc.com/comparch/article/91-04-088

> I'm not sure if allowing tagging was intentional when Motorola designed the 68k

Definitely not; it was a cost-saving measure, and a lot of software broke when a later processor made pointers fully 32-bit.

> but Mac OS did store flags in the upper bits of a pointer at some stage.

...and having a 32-bit-clean Finder was a big deal for a while because that meant it could run on the newer hardware.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by archangel_one on Wednesday January 04, 2012 @02:43PM (#3425542)

That seems like a slightly dubious choice to me, since unsigned long isn't guaranteed to be as long as a pointer; for example, under Microsoft's 64-bit compiler it's still only 4 bytes, but pointers are obviously 8. Possibly there is some extra preprocessor cleverness to deal with it, or maybe they don't expect it to work under that compiler?

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by ben0x539 on Wednesday January 04, 2012 @03:51PM (#3425844)

Yeah, there's preprocessor cleverness. It will choose either of unsigned, unsigned long or unsigned long long. There's a "#if defind HAVE_UINTPTR_T && 0" section, I'm curious why they're effectively disabling that.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by jof on Wednesday January 04, 2012 @02:58PM (#3425615)

Isn't this what C99's uintptr_t type tries to address?

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by archangel_one on Wednesday January 04, 2012 @03:16PM (#3425703)

Yes, exactly. I'm just a little curious since, without knowing anything about the code at all, unsigned long doesn't seem like a great choice. I'm sure the Ruby devs would know this though so I expect there must be some reason for it.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by bhousel on Wednesday January 04, 2012 @03:51PM (#3425842)

Here's the code: https://github.com/ruby/ruby/blob/trunk/include/ruby/ruby.h#...

It looks like uintptr_t support is there, just disabled for now.

[ Parent | on HN ]

Re:Never create Ruby strings longer than 23 characters by archangel_one on Wednesday January 04, 2012 @04:01PM (#3425877)

Thanks - the fallback to long long explains it for me.

[ Parent | on HN ]