News for hackers. Stuff that matters.
Re:What software engineers should know about search (2017) by karterk on Monday October 18, 2021 @03:22AM (#28903046)
A problem close to my heart. Good search is certainly still too difficult to pull off for small teams, and this was one of my motivations for building and open sourcing Typesense[1].

Most people think of search and immediately think of large data sets, but the problems that plague smaller datasets are equally interesting. It's less about performance and more about relevance. For e.g. searching across multiple fields for a compound query like "taylor swift style", requires breaking the query into segments (taylor swift | style) before searching for the appropriate fields. There are also a class of problems that traditional search engines that rely on BM25 or TF-IDF for ranking cannot reliably solve (e.g. searching on small texts like titles) where you have to consider distance between matching words (which TF-IDF and BM25 miss). Lastly, there is also personalization which is almost always left as an exercise to the reader :)

[1]: https://github.com/typesense/typesense

Re:What software engineers should know about search (2017) by slimsag on Monday October 18, 2021 @03:26AM (#28903075)
As a long time lurker of your work, and someone who works on a fair number of search engines myself - I'm curious, how well does TypeSense handle code search (punctuation, etc.)?
Re:What software engineers should know about search (2017) by karterk on Monday October 18, 2021 @03:35AM (#28903108)
Finally added support for indexing and/or separating on specific symbols in the ongoing pre-release builds. With that, I think Typesense should be able to handle code search, but I have never tried to index and search on code myself :)
Re:What software engineers should know about search (2017) by slimsag on Monday October 18, 2021 @04:44AM (#28903396)
Cool! I’d be keen to try it out sometime if you have docs on how I’d get it to index specific symbols
Re:What software engineers should know about search (2017) by karterk on Monday October 18, 2021 @04:55AM (#28903456)
Yes, please see here: https://github.com/typesense/typesense/issues/122#issuecomme...

The latest RC build is `0.22.0.rcs18`.

Re:What software engineers should know about search (2017) by carlreid on Monday October 18, 2021 @05:00AM (#28903480)
Typesense has been great for us so far. Easy to set up, works great with the simple queries that we need.

Lots of great additional functionality on top of search, for example security, with scoped API keys and the likes that we're looking forward to making use of.

Re:What software engineers should know about search (2017) by karterk on Monday October 18, 2021 @05:24AM (#28903636)
Thank you for your kind words. We are just getting started :)
Re:What software engineers should know about search (2017) by jimmyvalmer on Monday October 18, 2021 @07:58AM (#28904506)
This phrase "Thank you for your kind words" has in the last 1-2 years become the standard way people used to just say "Thanks". It sounds so cringey and mawkish.
Re:What software engineers should know about search (2017) by AtNightWeCode on Monday October 18, 2021 @04:23PM (#28911182)
For sure it depends on the quality of data and the target of the service. Basic knowledge in Elastic search will beat Google for confined data sets.

Major search engines have grown to the size where it is theoretical more beneficial to train a model and then query that model. In theory this will win but in the real world this will not work as we already can see with the failing Google search engine.

Re:What software engineers should know about search (2017) by remram on Monday October 18, 2021 @07:19PM (#28913074)
A limitation of TypeSense, and a lot of alternatives, is that they don't really work on Windows. TypeSense looks good for embedded situations (ie. inside an app) except it won't run on Windows.
Re:What software engineers should know about search (2017) by jabo on Monday October 18, 2021 @07:29PM (#28913167)
We don't publish a native Windows binary mainly due to the effort involved. However, I've heard from a couple of users that they've been able to use the Docker build or WSL to run the Linux binary on Windows machines.
Re:What software engineers should know about search (2017) by remram on Tuesday October 19, 2021 @12:17PM (#28919865)
Yeah I can't require users to set up Docker or WSL before running my application.
Re:What software engineers should know about search (2017) by XCSme on Tuesday October 19, 2021 @09:08AM (#28917654)
Wait, so this means I can not test it locally on my Windows machine?