Re:Ruby 1.9 made me remember how I hate the concept of encod...
by Tichy on Thursday May 13, 2010 @11:16AM
(#1344469)
I never understood how to do unicode in Ruby 1.8, though. Or I could never be sure would put it more succinctly. Especially not with Rails, somehow I could not even find anything about that via Google. It seemed to somehow work, but I want to know what is going on - is my stuff utf-8 or not.
[ Parent
| on HN ]
Re:Ruby 1.9 made me remember how I hate the concept of encod...
by carbon8 on Thursday May 13, 2010 @12:12PM
(#1344652)
As I understand it, Ruby 1.8 strings are simply treated as non-encoded sequences of bytes. It has some support for UTF-8, so as long as you can use UTF-8 and ISO-8859-1 (latin1, since the mapping is the same), just think about it as passing around byte sequences and don't need to do a lot of string manipulation, it's not horrible to work with. However, you do need to jump through hoops to use a number of the standard string methods like #length with multibyte characters (often, this means running the the string through #scan(/./mu) before working with it: http://blog.grayproductions.net/articles/bytes_and_character...). Rails also has a UTF-8 handler to help with manipulation of UTF-8 strings with multibyte characters, and I think it's on by default (http://api.rubyonrails.org/classes/ActiveSupport/Multibyte/H...)
Re:Ruby 1.9 made me remember how I hate the concept of encod...
by jamiecobbett on Friday May 14, 2010 @06:18AM
(#1346987)
In a sentence, Ruby 1.8 Strings are just a set of bytes, so methods like length count the number of bytes, Ruby 1.9 Strings are a set of characters, so length counts characters. This makes a difference when you have multi-byte characters, and want an accurate length, a correct split, reverse etc.
I found this series of articles well worth the time to read and understand: http://blog.grayproductions.net/articles/understanding_m17n
Re:Ruby 1.9 made me remember how I hate the concept of encod...
by Tichy on Thursday May 13, 2010 @08:34PM
(#1346148)
I know that Strings are just non-encoded sequences of bytes. But how do I know that/how Rails works? Ie does it use some regular expressions somewhere that might fail, or not fail? When I was new to Rails they added an extra method to strings to convert it to a kind of unicode string, but somehow later that was lost.
As I said, it seems to kind of work, but I would like to be sure what is going on.