mwsherman

I recommend starting with SWAR [1] before SIMD. Our registers are typically 64 bits, and one can try out SIMD patterns without taking a dependency on particular hardware.

This will only be effective if the data you’re working on is smaller than 64 bits. If you’re working with bytes, for example, you might get 8x parallelism.

[1] https://en.wikipedia.org/wiki/SWAR

While this article is about perf — and trading off semantic precision by design — there is a Unicode standard for sentence boundaries, may be interesting: https://www.unicode.org/reports/tr29/#Sentence_Boundaries

I implemented the sentence boundaries, but also thought that the notion of a “phrase” might be useful for such applications: https://github.com/clipperhouse/uax29/tree/master/phrases

Shameless plug, if one wishes to track down allocations in Go, an allocations explorer for VS Code: https://marketplace.visualstudio.com/items?itemName=Clipperh...

I’ve found C#’s frozen dictionary to be useful: https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

It’s optimized for fast reads in exchange for expensive creation.

That’s true, converting in either direction will typically allocate. Which it must, semantically.

One can use unsafe for a zero-copy conversion, but now you are breaking the semantics: a string becomes mutable, because its underlying bytes are mutable.

Or! One can often handle strings and bytes interchangeably with generics: https://github.com/clipperhouse/stringish

There is mention of how len() is bytes, not “characters”. A further subtlety: a rune (codepoint) is still not necessarily a “character” in terms of what is displayed for users — that would be a “grapheme”.

A grapheme can be multiple codepoints, with modifiers, joiners, etc.

This is true in all languages, it’s a Unicode thing, not a Go thing. Shameless plug, here is a grapheme tokenizer for Go: https://github.com/clipperhouse/uax29/tree/master/graphemes

In Go, string effectively serves as a read-only slice, if we are talking about bytes.

ReadOnlySpan<T> in C# is great! In my opinion, Go essentially designed in “span” from the start.

A Go allocations explorer for VS Code. Extension: https://marketplace.visualstudio.com/items?itemName=Clipperh...

Source: https://github.com/clipperhouse/go-allocations-vsix

Shameless plug, you may wish to do Lucene-style tokenizing using the Unicode standard: https://github.com/clipperhouse/uax29/tree/master/words

I move between Go and C#. I wrote a zero-allocation package in Go [1] and then ported to C# — and the allocations exploded!

I had forgotten, or perhaps never realized, that substrings in C# allocate. The solution was Spans.

Notably, it caused me to realize that Go had “spans” designed in from the start.

[1] https://github.com/clipperhouse/uax29

Putting on eng manager hat, the problem to solve is that this regression went undetected, not that Safari is slow.

The solution is a test that fails when Chrome and Safari have substantially different render times.

C# has implemented some SIMD for IndexOf: https://github.com/dotnet/runtime/pull/63285

Here’s a way to write tests next to the code: https://clipperhouse.com/go-test-csharp/

(Whether I recommend it, not sure! I did it and then undid it, with suspicion that tests were taking longer due to, perhaps, worse caching of build artifacts.)