News for hackers. Stuff that matters.

RealJon

Re:Ask HN: Are Lucene/Solr/ES Still Used for Search? on Monday July 22, 2019 @08:41AM (#20497412)
Predicate fields are indeed an oddity, but not an architectural one - it's for situations where the documents need to specify criteria (predicates) for when they should match - like only match for certain users, certain times of day etc. It's probably an underused feature imho since most people don't know this can be done efficiently.

If you have dynamic fields like in your SaaS example I recommend using a single map field rather than let data not under your control drive changes to the set of fields.

> Do you know how long a package update takes in Vespa, to add, say, a single field?

A few seconds. However, rather than having operators do any of this manually, set up an automatic process which deploys on each change made to the repo (i.e do CD).

> all the reference documentation in one place

https://docs.vespa.ai/documentation/api.html

Write speed (add or update) is typically between a few thousand to a few tens of thousands operations per second per node sustained, depending on sizeof data etc.

Sustaining throughput over long time is important and often overlooked mentioned in benchmarks.

No.
You don't need to plug in any Java code, you can use it with HTTP calls to read and write.

No plans to break it up into pieces (apart from already consisting of modules). It does one thing, it just happens to be a big thing :-)

If you have a mac of Linux box you can have it up and running in 10 minutes. Multi-node production deployments are no different because Vespa manages the nodes, not you directly.

No, this is run by a single, very cohesive, very remote team.

There is some misidentification in your list (our .def files are nothing to do with Teamcenter). We use 2 languages because we Java and C++ have different strengths which make each suitable at different layers of the architecture. The rest is a combination of "for good reasons" and "leftover scraps" :-)

TensorFlow on Spark is for learning, Vespa is for serving. Where Vespa excels is in evaluating a learned model very quickly over lots of documents. We're working on providing support for running models learned with TensorFlow directly. For now people make the translation on their own.
Not precisely sure where we were in 2011, but I think these are the biggest ones that came after, off the top of my head (i.e sure to be missing something):

  - Merging content and index clusters to one to make index clusters elastic and auto-recovering on data loss.
  - Fully realtime writes.
  - Support more advanced machine-learned ranking through tensors.
  - Streaming (personal) search supporting a large write rate.
  - Document references.
  - WAND and RANK operators.
  - Rank features over multivalue text fields.
  - Predicate fields.
  - Lots and lots of performance work.
Sorry about that - we haven't really optimized the module structure for newcomer comprehension. If you tell me what you want to are looking for I can point you to the right place.