News for hackers. Stuff that matters.
Re:How the embeddability of Lua impacted its design by jacques_chester on Monday December 03, 2012 @03:49AM (#4864553)
One of the classic FAQs about Lua is:

"Why doesn't Lua have unicode support?"

I mean, it does, but in the same fashion as ANSI C: you need to either pick a convention for multibyte characters or use a third-party library.

And that's not an accident. One part of Lua's embeddability design is that the standard implementation is written in pure ANSI C, and pure ANSI C ... doesn't support unicode.

Re:How the embeddability of Lua impacted its design by dalke on Monday December 03, 2012 @04:13AM (#4864597)
I don't understand how writing it in "pure ANSI C" prohibits Unicode support. Python, for example, is written in "pure ANSI C" and does support Unicode. Though doing so correctly and completely is hard, and requires code and data which bulks up what is otherwise a compact embeddable implementation.
Re:How the embeddability of Lua impacted its design by jacques_chester on Monday December 03, 2012 @05:14AM (#4864762)
Basically, it's as you guessed: Lua's string implementation is pretty much a transparent skin over C strings. The main difference is that Lua checks the boundaries and collects garbage.

It's a real turnoff for lots of people. In terms of "things new to Lua find unattractive", it's second only to 1-based indexing.

Re:How the embeddability of Lua impacted its design by zurn on Monday December 03, 2012 @03:57AM (#4864567)
Standard C has wchar_t, which is used for Unicode on platforms supporting Unicode.
Re:How the embeddability of Lua impacted its design by jacques_chester on Monday December 03, 2012 @06:41AM (#4864966)
wchar_t is implementation-specific.
Re:How the embeddability of Lua impacted its design by zurn on Tuesday December 04, 2012 @10:29AM (#4871326)
Sure. char is not specified to support ASCII either but it's not really a problem in practice :)

If you start avoiding all the things that are not fully specified by the standard, your C programs will end up doing very little indeed.

Now, it might be that some environments targeted by Lua have very spartan C runtimes: consoles. Anyone have a pointer to the C runtime docs for Xbox 360 and PS3?

Re:How the embeddability of Lua impacted its design by charliesome on Monday December 03, 2012 @06:36AM (#4864955)
wchar_t should never be used in any new code
Re:How the embeddability of Lua impacted its design by ygra on Monday December 03, 2012 @07:22AM (#4865059)
Just out of curiosity: What is its replacement? If I remember correctly it began its life as the char type for multi-byte character sets and later found use for UTF-(16|32) code units.

Should Unicode strings forcibly be UTF-8 in all programs and use char again (thus suffering from the same problem that Python 2 had where bytestrings and Unicode strings weren't sufficiently distinguished, often leading to programmers using the former for text)?

Or should you use short/int arrays instead and have no support from the stdlib at all for handling them (not that the stdlib is particularly good at handling text)?

Re:How the embeddability of Lua impacted its design by charliesome on Monday December 03, 2012 @07:24AM (#4865062)
If you're using UTF-8, use either uint8_t* (preferred) or char. If you're using UTF-32, use uint32_t. If you're using UTF-16, don't.

As far as library support goes, there are a few good Unicode libraries for C, most notably ICU4C.