Re:Anomaly Detection of Time Series Data Using Machine Learn...
by baldeagle on Thursday July 20, 2017 @02:24PM
(#14815066)
TLDR: This is a high level overview of all the terms in the topic (Anomaly, Time Series, Deep Learning). If you have a passing familiarity with the problem space, there is nothing new here.
There is an ad for xenonstack services at the bottom.
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by VHRanger on Thursday July 20, 2017 @02:25PM
(#14815076)
> This is because the oscillations are dependable upon the business cycle.
No, no, no, no. The "business cycle" is a misnomer; it's not a cycle. It doesn't fit any descriptions of a "cycle" except for the fact that it sometimes goes up and sometimes goes down.
Saying that it's a cycle implies you can time downward trends like crashes or predict their intensity, which, anyone in finance or maroeconomics will tell you is impossible
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by ares2012 on Thursday July 20, 2017 @08:05PM
(#14817199)
Business cycles are often a component of what is a composite function of the time series. So the time series itself might not have perfect cycles but the business cycle is usually one of the principal components of the time series, which is what I think they were referring to.
For example, the price of commodities does depend on the growing season (which is predictable) but other factors which are not cyclical so that the price itself is not easily predicted.
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by krona on Thursday July 20, 2017 @03:09PM
(#14815406)
Saying that it's a cycle implies you can time downward trends like crashes or predict their intensity, which, anyone in finance or maroeconomics will tell you is impossible
Just because something is hard doesn't mean it's impossible; You can predict anything with a degree of certainty. Then there are things like uncertainty and radical (Knightian) uncertainty.
Any chartered actuary would tell you the same if you could afford one, and they will know quite a bit about business cycles and how to model them, too.
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by VHRanger on Thursday July 20, 2017 @03:27PM
(#14815554)
Except you won't in macroeconomics. The simple problem in macroeconomics is lack of data -- history happened only once so you can't test counterfactuals.
You can try to build a model, but the confidence intervals for the next big crash are going to be either useless (way too wide) or bullshit if what you're trying to predict is the next market crash. Building a classifier around it machine learning style will not fundamentally help, either.
You can try it: here is an open source, serious, full scale macroeconomic model by the NY Fed [1]. Try to predict the next large crash with it and report back
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by VHRanger on Thursday July 20, 2017 @03:08PM
(#14815399)
Yes, there are seasonal variations in business (eg. agriculture!) but those aren't the "business cycle" which refers to the boom/bust recurrence (like the 1990s boom and 2000 tech bubble pop, the 2000-2008 boom and financial crisis, etc.)
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by VHRanger on Thursday July 20, 2017 @03:05PM
(#14815379)
noun: cycle; plural noun: cycles
1.a series of events that are regularly repeated in the same order.
So no, the "business cycle" is not a cycle. In the sense that you won't successfully fit a sin curve through the GDP that predicts timing or magnitude of significant deviations from the log GDP time series trend.
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by Xcelerate on Thursday July 20, 2017 @04:21PM
(#14815945)
If you're interested in more research level techniques, check out some of the work that's been done using minimum description length (MDL). It basically seeks to minimize the size of the model plus the size of the residual data (that the model doesn't predict) and uses Huffman compression as a proxy for the Kolmogorov complexity of a string:
Re:Anomaly Detection of Time Series Data Using Machine Learn...
by Omnipresent on Thursday July 20, 2017 @09:21PM
(#14817443)
Since this article is very high level, are there other resources that show how to apply some (or all) of these topics in actual code or on a sample set of data?
No, no, no, no. The "business cycle" is a misnomer; it's not a cycle. It doesn't fit any descriptions of a "cycle" except for the fact that it sometimes goes up and sometimes goes down.
Saying that it's a cycle implies you can time downward trends like crashes or predict their intensity, which, anyone in finance or maroeconomics will tell you is impossible
For example, the price of commodities does depend on the growing season (which is predictable) but other factors which are not cyclical so that the price itself is not easily predicted.
Just because something is hard doesn't mean it's impossible; You can predict anything with a degree of certainty. Then there are things like uncertainty and radical (Knightian) uncertainty.
Any chartered actuary would tell you the same if you could afford one, and they will know quite a bit about business cycles and how to model them, too.
You can try to build a model, but the confidence intervals for the next big crash are going to be either useless (way too wide) or bullshit if what you're trying to predict is the next market crash. Building a classifier around it machine learning style will not fundamentally help, either.
You can try it: here is an open source, serious, full scale macroeconomic model by the NY Fed [1]. Try to predict the next large crash with it and report back
[1] https://github.com/FRBNY-DSGE/DSGE.jl
The rest is left as an exercise to the reader.
"the recurrent cycle of harvest failure, food shortages, and price increases"
Can you predict the timing and magnitude of food shortages and prices?
"Discovering the intrinsic cardinality and dimensionality of time series using MDL" https://pdfs.semanticscholar.org/2049/50b3cd9cf2eef52f957df6...
A few papers that build on that one show how to incorporate anomaly detection (in the sense that the anomaly is incompressible).
Ahmad et al. "Unsupervised real-time anomaly detection for streaming data": http://www.sciencedirect.com/science/article/pii/S0925231217...
Anomaly detection benchmark (dataset and evaluation): https://github.com/numenta/NAB
Edit: supposed to be a light-hearted observation, I haven't read the article!