News for hackers. Stuff that matters.
Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by Aeolun on Monday August 01, 2022 @12:44AM (#32302165)
Fast, and ‘in-typescript’ seem to be mutually exclusive. At least for something as compute intensive as a full-text search engine.
Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by tgv on Monday August 01, 2022 @01:53AM (#32302538)
That's condescending. There is, obviously, a use case for in-core text search. E.g., mine is that it's better to waste a few CPU cycles in the browser than on the server. And Javascript runs pretty fast, and has been doing so for years now. And while a Javascript implementation won't beat a well-programmed C/Rust/Go version, the fact that it's in core helps it speed undoubtedly. So, if you ever need a web user to search some moderately sized set of texts, this might be really helpful.

As a matter of fact, I have a similar module in my applications, e.g. for typo resistant searching through help docs and hierarchical menus.

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by y4mi on Monday August 01, 2022 @05:30AM (#32303732)
> C/Rust/Go version

Minor nitpick: JS/V8 beats Go in most benchmarks that are relevant for search.

JS is the outlier here however, because of the insane amount of optimizations that made it perform so unreasonably well, despite being an interpreted language.

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by tgv on Monday August 01, 2022 @05:54AM (#32303850)
I've never seen such a benchmark. Which one is it?
Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by y4mi on Monday August 01, 2022 @07:15AM (#32304393)
It obviously depends on how you've implemented the feature, but there are a lot of cpmparitive benchmarks if you bother to put "go vs JS Performance benchmark" into the search bar and press enter. The language itself doesn't make code performance however, so you'll have good and bad implementations in any language you go with.

I.e. specifically regex, which is highly relevant in searching through strings: https://github.com/mariomka/regex-benchmark

And the always interesting techempower Project, which leaves the implementation to participants of each round. https://www.techempower.com/benchmarks/#section=data-r21&tes...

Choose whatever category you wish there, js is faster then go in almost all categories there.

Even though I said it before, I'm going to repeat myself as I expect you to ignore my previous message: the language doesn't make any implementation fast or slow. You can have a well performing search engine in go and JS. The performance difference will most likely not be caused by the language with these two choices. And the same will apply with C/Rust. The language won't make the engine performant and creating a maximally performant search engine is hard. But a theoretically perfect implementation would likely be fastest in C/Rust, followed by the usual suspects such as Go/Java/C#/JS and finally ending with all other interpreted languages such as ruby and python

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by tgv on Monday August 01, 2022 @07:46AM (#32304652)
Regex doesn't feature in this kind of library, at least not in mine. It's a token by token comparison/weighing, for which regular expressions are unsuitable. Mine looks at string prefixes and does a Levenshtein comparison on the results, and I think OP's project does that too.

The techempower benchmarks seem geared towards http backend server frameworks. There's only one Javascript framework that scores well, "just-js", but that's a bit low-level for a framework. I don't think it says much about text search performance.

> the language doesn't make any implementation fast or slow

Not ignoring that, but I think it's half true (you can't have a well performing app in native Python, basically, but a bad implementation will also cost a lot of performance). JS is fast enough for most tasks, that's true.

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by Aeolun on Tuesday August 02, 2022 @01:37AM (#32315474)
> the language doesn't make any implementation fast or slow

I think some languages are much easier to make things fast in than others. Even if the theoretical limit is the same (or nearly the same) in all languages.

Messing up somewhere in Javascript with async isn’t unlikely.

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by jahewson on Monday August 01, 2022 @12:45AM (#32302174)
Well, the types only exist at compile time, so they're fast at least!
Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by micheleriva on Monday August 01, 2022 @03:31AM (#32303139)
Disclaimer: Lyra author here!

The biggest problem I have with TypeScript is to know WHAT .ts files will be compiled into. I.e., if you're using enums or decorators, there's no plain support for them in JS so you'll end up having some kind of "polyfills", which are not so optimized.

Once you analyze your compiled JS, you can write TS knowing what you're gonna get, which is the trick to make it optimized.

My two cents :)

Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by nicoburns on Monday August 01, 2022 @04:26AM (#32303406)
The trick there is to not use enums or decorators! In all seriousness, I believe it’s common for typescript codebases to eschew these features, precisely because it’s hard to reason about the generated code (and preferable to avoid features that aren’t on an ECMAScript standards track.
Re:Lyra: Fast, in-memory, typo-tolerant, full-text search en... by Aeolun on Monday August 01, 2022 @07:39AM (#32304603)
Enums compile down to pretty standard objects right? Is there a situation in which you found them to do something odd?

Decorators, yeah, I can imagine that’d end up interesting.