Re:Show HN: Midas, a Streaming Anomaly Detector. Now Impleme...
by pmfgpmfg on Wednesday April 29, 2020 @09:59AM
(#23019275)
it's not clear to me how this API supports streaming edges. Looks like it takes lists of sources and destinations and loads them into memory.
[ Parent
| on HN ]
Re:Show HN: Midas, a Streaming Anomaly Detector. Now Impleme...
by steve0hh on Wednesday April 29, 2020 @10:53AM
(#23019976)
Thanks for your comments. I'll be updating the readme to better reflect it..
I've just added the midas struct, in which you can do the following:
```
m := midas.NewMidasModel(2, 769, 9460)
m.FitPredict(2,3,1) // inputs will be your src,dst and time (from your stream)
m.FitPredict(2,3,1) // same here
```
The MidasR model is next. :)
Re:Show HN: Midas, a Streaming Anomaly Detector. Now Impleme...
by floren on Wednesday April 29, 2020 @11:15AM
(#23020227)
Could you go into the selection criteria for those 3 numbers? I see 2 and 769 on the original MIDAS repo as default number of hash functions and buckets, but without explanation of why they're chosen. The third argument, "m", I don't see explained anywhere.
Re:Show HN: Midas, a Streaming Anomaly Detector. Now Impleme...
by steve0hh on Wednesday April 29, 2020 @11:28AM
(#23020376)
I adapted the code from the original paper from https://github.com/bhatiasiddharth/MIDAS/ .
And from one of the issues(https://github.com/bhatiasiddharth/MIDAS/issues/7#issuecomme...), M can be any reasonable value because it's just used to compute the edge hash (https://github.com/steve0hh/midas/blob/master/edgehash.go#L4...).
Hope it clarifies.
Re:Show HN: Midas, a Streaming Anomaly Detector. Now Impleme...
by siddhartb_ on Wednesday April 29, 2020 @12:03PM
(#23020782)
Hi, I'm the author of the MIDAS algorithm. We choose the number of hash functions and bucket according to the maximum error we can tolerate and the false positive probability theoretical guarantee we want. Please refer to the AAAI paper here: https://www.comp.nus.edu.sg/~sbhatia/assets/pdf/midas.pdf
Let me know if you need more details.