News for hackers. Stuff that matters.
Re:Search at Slack by jtoberon on Wednesday February 08, 2017 @01:45PM (#13600850)
Question for the author: how do you actually deploy your model? Do you have a dependency on Spark in your production system?
Re:Search at Slack by isabellat on Wednesday February 08, 2017 @02:44PM (#13601374)
Since its just a dot product between the learned weights and the feature vector, we do this in the application layer as lacksconfidence surmised.
Re:Search at Slack by lacksconfidence on Wednesday February 08, 2017 @01:55PM (#13600938)
Being that this is a SVM, which is typically evaluated as a simple linear sum of weights, I imagine they reimplemented that in the application layer. Would be curious how they handled the normalization steps (reimplement that as well?)
Re:Search at Slack by jliszka on Wednesday February 08, 2017 @03:13PM (#13601594)
Yep. We normalize our features as part of training, and the stdevs of each feature are part of the resulting model, along with the weights. (The means are always 0 because of the way we construct our training set.) The weights we use in production are actually normalized_weight / stdev.