News for hackers. Stuff that matters.
Re:Why Is Swift's String API So Hard? by zeveb on Friday November 06, 2015 @10:45AM (#10520016)
> Human-readable text, file paths, SQL statements, and others are all conceptually different, and this should be represented as different types at the language level. I think that having different conceptual kinds of strings be distinct types would eliminate a lot of bugs. I'm not aware of any language or standard library that does this, though.

The Common Lisp standard includes pathname objects, with an implementation-defined (since different platforms have different conventions) mapping from strings to path objects. It doesn't specifically address the type-safety issue, though, since just about every standard function which accepts a pathname will also accept a namestring.

There exist libraries such as CLSQL which enable representation of SQL queries as objects.

Common Lisp: doing it right for so long that most people haven't even heard of it!

Also:

> When measuring a string to ensure that it fits in a 140-character tweet, you want to go by unicode code points.

Really? Doesn't one actually want 140 graphemes? Regardless of what one wants, does Twitter enforce 140 code points, or 140 bytes?

It seems to me that graphemes = characters, and that the fact that some characters are made up of multiple codepoints is as important as the fact that some code points are made up of multiple bytes.

Re:Why Is Swift's String API So Hard? by TazeTSchnitzel on Friday November 06, 2015 @11:38AM (#10520336)
> Regardless of what one wants, does Twitter enforce 140 code points, or 140 bytes?

Number of code points after normalisation: https://dev.twitter.com/overview/api/counting-characters

Re:Why Is Swift's String API So Hard? by grey-area on Friday November 06, 2015 @11:01AM (#10520126)
Isn't this quite common? Most languages allow you to represent those strings as different types and parse them/to from a plain string representation.

From Ruby: String, Pathname, Url, ActiveRecord::Relation etc or from Go: strings, database/sql stmt + orm types built on top, net/url etc though file paths are just strings.

Re:Why Is Swift's String API So Hard? by MrBuddyCasino on Friday November 06, 2015 @11:29AM (#10520283)
It is, even Java does that.

Where it gets interesting is advanced trickery, as performed by Haskell. You can have a type String that cares of escaping/unescaping stuff that comes from tainted sources in a webapp, eliminating a whole lot of security issues. Thats just one example I know, but I'm not a Haskell guy - there are surely more.

Re:Why Is Swift's String API So Hard? by mozumder on Saturday November 07, 2015 @05:57AM (#10524429)
Yep. He basically just described "classes" in the object-oriented world.
Re:Why Is Swift's String API So Hard? by tokenrove on Saturday November 07, 2015 @09:07AM (#10524815)
I don't think it has much to do with classes. Python, for example, has no pathname/string distinction that I know of.

Common Lisp is in the object-oriented world, and indeed some of the things mentioned are implemented as classes. I think the point was more about the standard library's attempt to do The Right Thing and how library implementors tend to follow that.

(Also, for better or for worse, CL's pathnames contain information like host, device, and version, that aren't so relevant on the major OSes in use these days; but try dealing with VMS filenames with Ruby's Pathname...)

Re:Why Is Swift's String API So Hard? by lispm on Saturday November 07, 2015 @06:29PM (#10526657)
Device and even host are quite common in the Windows world:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

Volume = Device

Or Microsoft's UNC path:

https://en.wikipedia.org/wiki/Path_(computing)#Uniform_Namin...

\\ComputerName\SharedFolder\Resource -> \\host\device\path

Re:Why Is Swift's String API So Hard? by kazinator on Friday November 06, 2015 @11:09AM (#10520173)
> implementation-defined (since different platforms have different conventions) mapping from strings to path objects.

Unfortunately, this translates to a problem. We don't have that many path representations any more, but Common Lisp implementations for the same platform (or platform family, like POSIX) do the translation differently.

If you want to work with CL pathnames, but expose paths as strings to the user, and use multiple CL implementations, you have to write your own path translation which behaves the same way.

For instance, what is

  a/b/c/
? Is that a pathname with directory a b c, and an empty name? Or directory a b and name c?

Do you map suffixes to types? How about compounded suffixes like .tar.gz? Is that a "tar.gz" type or a "gz" type?

Re:Why Is Swift's String API So Hard? by wnoise on Friday November 06, 2015 @05:28PM (#10522496)
Are there any filesystems that support empty names? If not, clearly the second is correct.
Re:Why Is Swift's String API So Hard? by Retra on Friday November 06, 2015 @11:35AM (#10520316)
Ideally, you'd provide interfaces for both interpretations.
Re:Why Is Swift's String API So Hard? by kazinator on Friday November 06, 2015 @12:12PM (#10520521)
Two interpretations across N features: (expt 2 N) interface combinations. :)
Re:Why Is Swift's String API So Hard? by Retra on Friday November 06, 2015 @12:30PM (#10520630)
That's really only a problem if you're building interfaces that can't be composed.
Re:Why Is Swift's String API So Hard? by zzalpha on Friday November 06, 2015 @11:00AM (#10520121)
I wonder if part of the reason for this is that Common Lisp dates back to a time when filesystem path structures were a lot more variable.

These days, you either have Unix-style or Windows-style paths and that's basically it

But go back 30 years and there was quite a bit more variety that would make an abstraction layer all but a requirement if you wanted to define a portable language spec.

Re:Why Is Swift's String API So Hard? by gongador on Sunday November 08, 2015 @02:28PM (#10529497)
Indeed.

"The dominant file systems at the time the design was done were TOPS-10, TENEX, TOPS-20, VAX VMS, AT&T Unix, MIT Multics, MIT ITS, not to mention a bunch of mainframe OS's. Some were uppercase only, some mixed, some were case-sensitive but case-translating (like CL). Some had dirs as files, some not. Some had quote chars for funny file chars, some not. Some had wildcards, some didn't. Some had :up in relative pathnames, some didn't. Some had namable root dirs, some didn't. There were file systems with no directories, file systems with non-hierarchical directories, file systems with no file types, file systems with no versions, file systems with no devices, and so on. People often critique the design now because Unix and Mac and DOS are so comparatively similar (compared to some of those others)..."

-- Kent Pitman in comp.lang.lisp, https://groups.google.com/forum/#!original/comp.lang.lisp/Pl...

Re:Why Is Swift's String API So Hard? by rprospero on Friday November 06, 2015 @12:46PM (#10520737)
I remember reading a LISP book from the 80's that gently bragged about how all the legacy C code would have to be rewritten after upcoming operating systems moved beyond identifying files by a tree of folders, but that your LISP code would be future proof.
Re:Why Is Swift's String API So Hard? by jdmichal on Friday November 06, 2015 @02:09PM (#10521310)
This strikes me as one of those instances were the cost of the abstraction exceeds the savings. There are some things which are simply core to your code. Attempting to abstract those things is going to create a lot of pain when, realistically, those things changing would require component rewrites anyway.

If you're working with files, you've probably made many implicit assumptions about how the file system works and how it is represented. So you are either going to have a leaky abstraction that works, or a non-leaky abstraction that has those assumptions built in and will break when porting to a system without those assumptions anyway.

Re:Why Is Swift's String API So Hard? by mikeash on Friday November 06, 2015 @11:24AM (#10520257)
I'm actually not sure what Twitter is doing, now. Putting Emoji into the box decreases the remaining character count by one for each Emoji. But putting Zalgo into the box decreases the remaining character count by several for each grapheme cluster. My head hurts.
Re:Why Is Swift's String API So Hard? by TazeTSchnitzel on Friday November 06, 2015 @11:39AM (#10520342)
Twitter actually counts codepoints, but after normalisation, so it doesn't matter which of the two ways you represent é: https://dev.twitter.com/overview/api/counting-characters
Re:Why Is Swift's String API So Hard? by mikeash on Friday November 06, 2015 @11:52AM (#10520410)
Right, stupid me, emoji are still a single code point. Put a flag in the box, and the count decreases by two.
Re:Why Is Swift's String API So Hard? by TazeTSchnitzel on Friday November 06, 2015 @03:44PM (#10521899)
Well, depends on the emoji. Some use combing characters, e.g. a red heart is the black heart character plus some invisible joiner IIRC, and some of the family emoji are composed of their individual components (mom/dad/girl/boy) with joiners.
Re:Why Is Swift's String API So Hard? by mikeash on Friday November 06, 2015 @04:22PM (#10522125)
Yes, you're right. In a fit of silliness I was testing only with the ones that are a single code point, though.
Re:Why Is Swift's String API So Hard? by zuck9 on Friday November 06, 2015 @11:12AM (#10520191)
Twitter has a limit of 140 characters to fit in a regular SMS (the rest 20 are used for the username).

From https://en.wikipedia.org/wiki/Short_Message_Service

> Short messages can be encoded using a variety of alphabets: the default GSM 7-bit alphabet, the 8-bit data alphabet, and the 16-bit UCS-2 alphabet. Depending on which alphabet the subscriber has configured in the handset, this leads to the maximum individual short message sizes of 160 7-bit characters, 140 8-bit characters, or 70 16-bit characters.

Re:Why Is Swift's String API So Hard? by richardwhiuk on Friday November 06, 2015 @11:16AM (#10520218)
That's not true. Firstly, a single tweet doesn't necessarily fit into a single SMS (as the Twitter limit is per character, whereas an SMS is limited per byte).

Secondly, an SMS is limited to 140 bytes, which is 160 7 bit 'characters', or 140 8 bit 'characters' or 70 16 bit 'characters' (in UCS 2, normally).

Re:Why Is Swift's String API So Hard? by mikeash on Friday November 06, 2015 @11:26AM (#10520270)
It was true. Twitter heavily emphasized SMS back when it was new.

The 140-thingy limit hasn't been about SMS for a while now, and has just transitioned into a weird artificial limitation that's become part of what Twitter is. But that is how it started out.

Re:Why Is Swift's String API So Hard? by moron4hire on Friday November 06, 2015 @11:18AM (#10520232)
That might be the origin, but that is not the current implementation. For example, URLs only ever count as 23 letters, because they always get shortened using Twitter's own t.co shortener. BUT they still display in their original form, so unless they are de-referencing the shortened URL on the fly, they have to be storing what the user typed and a pre-processed form of the text are being stored. That's a lot more than 140 ASCII characters.
Re:Why Is Swift's String API So Hard? by mikeash on Friday November 06, 2015 @01:19PM (#10520964)
What's really fun is that URLs always count as 23 characters even if they're shorter. For example, I had a tweet about Apple's Keynote.app fail to send the other day, because Twitter thought "Keynote.app" was a URL and "shortened" it such that my tweet exceeded the length limit afterwards. Fun times.
Re:Why Is Swift's String API So Hard? by moron4hire on Friday November 06, 2015 @03:15PM (#10521728)
Yeah, it's not good. I think Twitter would do well to not hurt users with things like that. Use the restraint of UI to keep people's posts small. It's difficult to type a screed into a single-line textbox. But get rid of any actual character limit at all.