BayesDB: A probabilistic programming platform

Re:BayesDB: A probabilistic programming platform by nightski on Sunday July 09, 2017 @04:45PM (#14731578)

I have not, so take my words with a grain of salt. However, with only 6 dimensions 100 trillion vectors is probably extremely overkill (unless you are talking time series data?). Probabilistic programming, from what I understand, thrives on situations where you do not have a lot of data (instead of an overabundance).

You could probably achieve very interesting results by taking a much smaller sample from your large data set.

[ Parent | on HN ]

Re:BayesDB: A probabilistic programming platform by afpx on Sunday July 09, 2017 @06:59PM (#14732191)

Could be. I'm just a software engineer with a high pain tolerance and a lot of tenacity who is trying to make conclusions from lots of error-ridden data. It seems difficult to find experts in this space. I've interviewed several statistics Ph.d. from reliable schools, but no luck yet. By the way, sampling doesn't seem to help because all of the data is relational. Or, maybe I don't know how to sample it effectively.

Anyway, in my case, I have data feeds, but none of them are 100% reliable. There is error, and I can guess the error. I want to infer things from the data, but I know that the conclusions are unreliable. So, I want to know how unreliable my conclusions are, if that makes sense.

Anyway, I'm an amateur here. But, My independent research led me to things like MCMC and probabilistic programming which allows me to model things better.

[ Parent | on HN ]