News for hackers. Stuff that matters.
Re:American Flag Sort by karmakaze on Tuesday April 23, 2024 @08:22PM (#40138982)
At first I didn't think I'd be interested.

> Using some efficiency techniques, it is twice as fast as quicksort for large sets of strings.

Re:American Flag Sort by hinkley on Tuesday April 23, 2024 @09:49PM (#40139541)
I’m not calling it flag sort.

It’s an in-place bucket sort which is interesting.

Re:American Flag Sort by wodenokoto on Wednesday April 24, 2024 @01:21AM (#40140870)
From Wikipedia[1]:

> The name American flag sort comes by analogy with the Dutch national flag problem[2] in the last step: efficiently partition the array into many "stripes".

[1] https://en.m.wikipedia.org/wiki/American_flag_sort

[2] https://en.m.wikipedia.org/wiki/Dutch_national_flag_problem

Re:American Flag Sort by hinkley on Wednesday April 24, 2024 @03:09AM (#40141529)
I read this rationale before I made my comment.
Re:American Flag Sort by kleiba on Wednesday April 24, 2024 @01:42PM (#40147448)
> From Wikipedia[1]:

...or from the very article itself...

Re:American Flag Sort by remram on Wednesday April 24, 2024 @10:44AM (#40144992)
Or in-place radix sort maybe.

I agree with you, the name is ridiculous, reminds me of miracle sort or intelligent design sort: https://www.dangermouse.net/esoteric/intelligentdesignsort.h...

Probably those two flag sort algorithms were named before the joke algorithms were popularily known.

Re:American Flag Sort by thefifthsetpin on Wednesday April 24, 2024 @08:36AM (#40143627)
Quicksort isn't great when comparing two items is expensive.
Re:American Flag Sort by muti on Tuesday April 23, 2024 @07:54PM (#40138782)
Re:American Flag Sort by AnarchismIsCool on Tuesday April 23, 2024 @08:30PM (#40139032)
Looks more like anarchist flag sort...
Re:American Flag Sort by codetrotter on Tuesday April 23, 2024 @09:18PM (#40139346)
I’ll take your word for it, AnarchismIsCool ;)
Re:American Flag Sort by aaron695 on Wednesday April 24, 2024 @12:08AM (#40140462)
> Visualisation https://www.youtube.com/watch?v=k1XkZ5ANO64

OT: I quite liked the follow up video Youtube suggested -

Sorting algorithms to relax/study to - https://www.youtube.com/watch?v=vr5dCRHAgb0

Re:American Flag Sort by peterleiser on Wednesday April 24, 2024 @12:31AM (#40140601)
The next video that came up for me was "sorting algorithms to relax/study to": https://www.youtube.com/watch?v=vr5dCRHAgb0

I'm almost embarrassed by how long I've watched it!

Re:American Flag Sort by knodi123 on Wednesday April 24, 2024 @01:30AM (#40140932)
No offense, but I don't think that visualization helps very much.

Here's a visualization of a much larger data set that illustrates what's actually happening with the buckets and the major steps.

https://www.youtube.com/watch?v=21jEd1FUiV8

Re:American Flag Sort by fourteenfour on Wednesday April 24, 2024 @02:39AM (#40141364)
Thanks, this one makes the stripe pattern much more apparent.
Re:American Flag Sort by tomphoolery on Tuesday April 23, 2024 @08:01PM (#40138819)
Freedom Sort.
Re:American Flag Sort by tomschlick on Tuesday April 23, 2024 @08:56PM (#40139220)
Cue the Team America theme song
Re:American Flag Sort by mcfedr on Wednesday April 24, 2024 @02:26AM (#40141298)
That is basically what I hear in my head everytime I see that flag
Re:American Flag Sort by tantalor on Tuesday April 23, 2024 @09:16PM (#40139336)
For democracy!
Re:American Flag Sort by seattle_spring on Tuesday April 23, 2024 @11:43PM (#40140295)
I'm doing my part!
Re:American Flag Sort by __d on Wednesday April 24, 2024 @02:23AM (#40141274)
would you like to know more?
Re:American Flag Sort by tom_ on Tuesday April 23, 2024 @09:21PM (#40139358)
If you disagree, see the first amendment! If you still disagree, see the second amendment!
Re:American Flag Sort by mondobe on Tuesday April 23, 2024 @11:30PM (#40140213)
And if you STILL disagree, see the third amendment! If you're a soldier, in my house, I guess.
Re:American Flag Sort by owlninja on Tuesday April 23, 2024 @08:32PM (#40139054)
The 'more information' site linked on the page seems like something some HN folk love.

https://www.workmall.com/flags/united_states_flag.html

Re:American Flag Sort by defrost on Tuesday April 23, 2024 @08:42PM (#40139136)
There's a Peter M. McIlroy as well!?!

All this time and I thought it was just Doug, I'm guessing Peter might be the son of the father of pipe?

Re:American Flag Sort by icsa on Tuesday April 23, 2024 @10:05PM (#40139643)
Yes. He is a software engineer in Seattle.
Re:American Flag Sort by jdeisenberg on Tuesday April 23, 2024 @09:09PM (#40139287)
Wondering how this sort would perform with Unicode input (given 256 buckets); specifically what the results would look like.
Re:American Flag Sort by dhosek on Tuesday April 23, 2024 @10:54PM (#40139963)
Ignoring the fact that if you have Unicode encoded strings, you probably want some sort of lexical sort rather than a sort on the raw Unicode character codes which is semantically meaningless, there is no reason to assume that Unicode data would be any different to sort than any other data.
Re:American Flag Sort by bawolff on Tuesday April 23, 2024 @09:39PM (#40139474)
Presumably it would be fine, since unicode input is still a string of bytes at the end of the day.
Re:American Flag Sort by moonchild on Tuesday April 23, 2024 @10:32PM (#40139825)
Unicode collation is very complex. If you wanted to sort according to that, then you would need to devise an isomorphism between unicode strings and bitstrings such that lexicographic ordering in the bitstring domain agrees with unicode collation in the unicode domain. I'm not saying it's impossible, but it's not at all obvious how to do it. On the other hand, if all you want is some total order for acceleration structures, then you can indeed just use an arbitrary encoding like utf8 and do the obvious thing.
Re:American Flag Sort by dsamarin on Wednesday April 24, 2024 @12:00AM (#40140409)
The obvious solution to this is to pre-compile a list of all possible Unicode strings up to the required length, sort them, and create a lookup table using their indices. I wonder for what kind of project this would be useful.
Re:American Flag Sort by thfuran on Wednesday April 24, 2024 @09:16AM (#40144022)
If that works for you for anything but very small strings, I want to buy your computer.
Re:American Flag Sort by remram on Wednesday April 24, 2024 @02:06PM (#40147692)
Even for 3-codepoint strings it already wouldn't fit in any computer that exists. And that is not sufficient to encode all 1-glyph strings.
Re:American Flag Sort by IncreasePosts on Wednesday April 24, 2024 @12:06AM (#40140447)
Isn't that exactly what the Unicode collation algorithm is?

https://unicode.org/reports/tr10

tl;dr from its wikipedia page:

> The Unicode collation algorithm (UCA) is an algorithm defined in Unicode Technical Report #10, which is a customizable method to produce binary keys from strings representing text in any writing system and language that can be represented with Unicode. These keys can then be efficiently compared byte by byte in order to collate or sort them according to the rules of the language, with options for ignoring case, accents, etc

Re:American Flag Sort by moonchild on Wednesday April 24, 2024 @04:09PM (#40148966)
Ah! You are right.
Re:American Flag Sort by bawolff on Wednesday April 24, 2024 @12:48AM (#40140695)
You can just use the icu library to do that. Its a very common thing to do when working with unicode collations.
Re:American Flag Sort by nanidin on Tuesday April 23, 2024 @09:42PM (#40139492)
It says it sorts one byte at a time. I think this would break for anything not utf-8.
Re:American Flag Sort by brirec on Wednesday April 24, 2024 @12:17AM (#40140521)
This would even break UTF-8, since multi-byte characters are a thing!
Re:American Flag Sort by ursusmaritimus on Wednesday April 24, 2024 @04:25AM (#40141929)
Lexicographic encoding of UTF-8 byte sequences matches lexicographic order of the sequence of Unicode code-points. So you can sort UTF-8 strings as byte strings. Not that sorting by code-points has much meaning, but you can use the Unicode collation algorithm first.
Re:American Flag Sort by remram on Wednesday April 24, 2024 @02:08PM (#40147712)
TIL, thanks. That makes sense given how the length prefixes look like but I never thought about it. I wonder if this was by chance or if the UTF-8 creator thought about it.
Re:American Flag Sort by mzs on Wednesday April 24, 2024 @04:35PM (#40149267)
% printf "%s\n" A B | sort

A

B

% printf "%s" A B | xxd -b -c4

00000000: 11110000 10011101 10010000 10110100 ....

00000004: 11110000 10011101 10010000 10110101 ....

% printf "%s" A B | xxd -c1 -ps | sort | xxd -r -ps | xxd -b -c4

00000000: 10010000 10010000 10011101 10011101 ....

00000004: 10110100 10110101 11110000 11110000 ....

% printf "%s" A B | xxd -c1 -ps | sort | xxd -r -ps

????????

Re:American Flag Sort by dumbo-octopus on Wednesday April 24, 2024 @12:51AM (#40140719)
How would that break anything? The strings aren't being split.
Re:American Flag Sort by EmilyHughes on Wednesday April 24, 2024 @01:18AM (#40140851)
UTF-8 is downward compatible to ASCII, so anything that is just a standard character (like every character in this comment) is just a byte.
Re:American Flag Sort by nanidin on Thursday April 25, 2024 @10:04PM (#40165201)
Indeed, and is à less than b? Not in Unicode!
Re:American Flag Sort by ts4z on Tuesday April 23, 2024 @09:58PM (#40139606)
Seems like it should work for arbitrary byte strings (any charset, any encoding)but obviously the performance characteristics will differ because of non-uniform distribution. But that happens even in ASCII.
Re:American Flag Sort by nanidin on Thursday April 25, 2024 @10:05PM (#40165208)
Yes, you’ll get something sorted based on the bytes in the string but it won’t be lexicographically correct - for example, à will be sorted after b.
Re:American Flag Sort by seanhunter on Wednesday April 24, 2024 @12:42AM (#40140660)
I think unicode is intrinsically antithetical to freedom. This would do great with ASCII.
Re:American Flag Sort by re on Tuesday April 23, 2024 @11:00PM (#40139998)
Looks like there's a Rust implementation: https://crates.io/crates/afsort

> When sorting English words, this implementation seems to be about 40% faster than sort_unstable from the Rust standard library.

That's better than I would have expected. It would be interesting to see how it does on other types of string sort problems.

Re:American Flag Sort by antonhag on Wednesday April 24, 2024 @02:48AM (#40141423)
Hi! Crate author here, happy to see my old project getting mentioned. Let me know if you have any questions!
Re:American Flag Sort by ewalk153 on Wednesday April 24, 2024 @06:13AM (#40142572)
Do you actively use this function in any projects? What was your inspiration to write the crate?
Re:American Flag Sort by antonhag on Wednesday April 24, 2024 @12:28PM (#40146365)
I don't actively use it, unfortunately. The main inspiration was to faster sort inputs into the https://github.com/BurntSushi/fst crate, which I in turn used to try to build a search library.
Re:American Flag Sort by Zelizz on Wednesday April 24, 2024 @12:35AM (#40140624)
Pretty much just a variation of counting sort with a worse name? https://en.wikipedia.org/wiki/Counting_sort
Re:American Flag Sort by dumbo-octopus on Wednesday April 24, 2024 @12:43AM (#40140666)
No. The very article you link details the difference, and that Counting Sort is often a subroutine in Radix Sort, and that Counting Sort is extremely poorly suited to strings, which is the entire purpose of this sort. And this name is better. So... "no" in basically every way possible.
Re:American Flag Sort by Zelizz on Wednesday April 24, 2024 @10:56AM (#40145127)
Characters in a string can be thought of as digits in a base-256 number. Call counting sort recursively on each bucket, looking at the next character in the string. Can you not see the similarity?
Re:American Flag Sort by chowells on Wednesday April 24, 2024 @11:53AM (#40145892)
There is no one who fails to see the similarity, because they're both kinds of radix sorts. For instance, a counting sort is a degenerate form of radix sort that only does one pass.

But where the classic radix sort is bottom-up, the American flag sort is top-down. That really matters for data where the the most significant portions of the data is likely to be all you need to consider for the sorting. While a top-down sort will have similar performance to bottom-up in the worst case, in the best case it can skip a lot of passes, especially if the inputs are much longer than the longest common prefix.

Re:American Flag Sort by dumbo-octopus on Wednesday April 24, 2024 @02:22PM (#40147847)
They're related, but in a very specific way that does not meet the bar for "variation" in my opinion. As you described, counting sort is a very simple procedure that works well only on a very specific input domain. If another algorithm has "more logic" than counting sort itself in order to transform a very different input domain into a format that is suited for making many repeated calls to (a procedure that may be implemented as) counting sort, in a specific pattern, and appropriately coalescing the results of those calls, I think it is appropriate to let it have its own name. Would you prefer to refer to every possible utilization of counting as "that version of counting sort where...."?
Re:American Flag Sort by carabiner on Wednesday April 24, 2024 @12:36AM (#40140631)
A superior name.
Re:American Flag Sort by sltkr on Wednesday April 24, 2024 @02:46PM (#40148065)
American flag sort is essentially an in-place version of counting sort.

That's talking about the single-pass American flag sort. If you use American flag sort to sort strings, you do multiple passes: on the first pass, you sort all strings by their first character, then for each starting character, you sort the strings with that starting character (which are now in a contiguous subarray) by their second character, and so on.

@chowells calls it a “top-down radix sort” below, which is a great description. It also explains the strengths and weaknesses of the two algorithms: radix sort works great for small strings of fixed length (e.g. IPv4 addresses, which can be thought of as 4-byte sequences) while American flag sort works great for variable-length strings like actual textual strings, especially if they don't share common prefixes (e.g. dictionary words, usernames, etc.)

@hinkley pointed out that the recursive version is just a bucket sort, but in-place. Which is also true!

tl;dr:

Single-pass American flag sort = in-place counting sort

Recursive American flag sort = in-place bucket sort

Re:American Flag Sort by Someone on Wednesday April 24, 2024 @01:42AM (#40140993)
The paper (https://static.usenix.org/publications/compsystems/1993/win_... linked to from the page) is well-written and worth reading, iterating over multiple C programs that build up to this sort algorithm and giving a good rationale for each iteration.
Re:American Flag Sort by awanderingmind on Wednesday April 24, 2024 @05:51AM (#40142409)
There is currently a typo in the link which prevents it from working - the correct link is https://static.usenix.org/publications/compsystems/1993/win_...

Interesting read, thanks!

Re:American Flag Sort by Traubenfuchs on Wednesday April 24, 2024 @02:34AM (#40141335)
> it is twice as fast as quicksort for large sets of string

Define large.

Re:American Flag Sort by TheRealPomax on Wednesday April 24, 2024 @03:37PM (#40148626)
They did. But you'll need to read the (free) paper [1] to get the details omitted in the NITS catalog summary.

[1] http://www.usenix.org/publications/compsystems/1993/win_mcil...

Re:American Flag Sort by Oarch on Wednesday April 24, 2024 @04:08AM (#40141855)
The comments made me realise there's no national flag with lots of vertical stripes. Seems like a hole in the market.

Someone alert CGP Grey!

Re:American Flag Sort by Kon-Peki on Wednesday April 24, 2024 @09:16AM (#40144026)
Sorts like these work really well on GPUs. You would start with a single thread for the first pass and then keep spawning one-thread-per-bucket for every additional pass until done. It isn't particularly taxing on the GPU; your advantage comes from being able to run potentially thousands of threads simultaneously without having to worry about coordinating read/write memory access.
Re:American Flag Sort by NikkiA on Wednesday April 24, 2024 @10:12AM (#40144617)
This strikes me that it should parallelize pretty well if you deputise the individual stripes to multiple threads.
Re:American Flag Sort by franze on Wednesday April 24, 2024 @03:32PM (#40148569)
today i coded franzSort, faster than mergeSort, slower than quickSort

function franzSort(a) { function m(l, r) { let s = []; while (l.length && r.length) { if (l[0] <= r[0]) { s.push(l.shift()); } else { s.push(r.shift()); } } return s.concat(l).concat(r); }

    if (a.length <= 1) {
        return a;
    }
    let h = Math.floor(a.length / 2);
    return m(franzSort(a.slice(0, h)), franzSort(a.slice(h)));
}

// Example usage let arr = [5, 3, 8, 6, 2, 7, 1, 4]; let sortedArr = franzSort(arr); console.log(sortedArr);

Re:American Flag Sort by jon_richards on Wednesday April 24, 2024 @04:36PM (#40149274)
The stars in the American flag aren't used. This is clearly Pride flag sort.
Re:American Flag Sort by oniony on Wednesday April 24, 2024 @05:57PM (#40150246)
And Mauritius's flag has more stripes.