News for hackers. Stuff that matters.
Re:China Built the World’s Largest Telescope, Then Came the ... by xevb3k on Sunday August 26, 2018 @11:46PM (#17849316)
“At other telescopes, astronomers are developing machine-learning algorithms that could identify, extract, and compensate for dirty data“

Whenever I see “machine-learning” in the context of scientific development I just mentally replace it with “magic”.

Employing machine learning, at the data acquisition stage, is a really bad idea. It can easily introduce biases...

Re:China Built the World’s Largest Telescope, Then Came the ... by oh-kumudo on Monday August 27, 2018 @12:30AM (#17849437)
I mean de-noising techniques, nowadays can be definitely claimed as Machine Learning, even though they have long existed prior to even ML as a concept even being invented...
Re:China Built the World’s Largest Telescope, Then Came the ... by wadkar on Monday August 27, 2018 @01:26AM (#17849612)
I think the defining characteristic if machine learning system is that they get better as they experience more (see more examples). The question then the ask is two fold (a) do de-nosing techniques improve when they see more samples? (b) do the application of ML techniques applied in this context mathematically equivalent?
Re:China Built the World’s Largest Telescope, Then Came the ... by hikarudo on Monday August 27, 2018 @06:29AM (#17850551)
That is only the case for online learning systems. Offline learning is definitely machine learning.
Re:China Built the World’s Largest Telescope, Then Came the ... by dsfyu404ed on Monday August 27, 2018 @09:16AM (#17851280)
You can "de-noise" all you want and you're still gonna get crap compared to what it would be without the noise. It's like the 3d RF equivalent of looking for ripples made by water striders in the middle of a jet ski race.
Re:China Built the World’s Largest Telescope, Then Came the ... by skybrian on Monday August 27, 2018 @11:16AM (#17851971)
Seems like it depends on the noise. If it didn't work for removing atmospheric distortion, they wouldn't be building bigger telescopes.
Re:China Built the World’s Largest Telescope, Then Came the ... by natmaka on Monday August 27, 2018 @03:17AM (#17849931)
Many nowadays ML (and even "IA")-dubbed techniques are in fact borrowed from good ole operational research.
Re:China Built the World’s Largest Telescope, Then Came the ... by gumby on Monday August 27, 2018 @12:42PM (#17852519)
Indeed, core memory was invented by Jay Forester for is operational research in city planning!
Re:China Built the World’s Largest Telescope, Then Came the ... by xevb3k on Monday August 27, 2018 @01:17AM (#17849579)
For example?
Re:China Built the World’s Largest Telescope, Then Came the ... by tanilama on Monday August 27, 2018 @02:36AM (#17849799)
PCA for example. It is essentially SVD. And it is quite funny though, I believe that there is no learning in PCA(maybe depends on implementation, but the popular one I believe used heuristics), it is still categoried as ML techniques all the time.
Re:China Built the World’s Largest Telescope, Then Came the ... by chriswarbo on Monday August 27, 2018 @06:50AM (#17850615)
PCA can "learn" in the sense that we can apply it to more and more data, and that can affect the directions of the components. For example, let's say we take measurements one at a time and apply PCA to the whole dataset after each measurement; if the first 100 measurements follow some distibution A, then PCA will give the components of A; if the next 1000 measurements follow some different distribution B, then PCA will 'learn' to give the components of B a higher priority than those of A; i.e. it adapts to the data.

In that hypothetical setup, we end up storing every raw datapoint. If we think of these as parameters, then PCA is a non-parametric model; although it's maximally inefficient in terms of information/compression.

Re:China Built the World’s Largest Telescope, Then Came the ... by plouffy on Monday August 27, 2018 @12:45PM (#17852542)
The learning aspect could be how many principal components to use ?
Re:China Built the World’s Largest Telescope, Then Came the ... by tanilama on Monday August 27, 2018 @01:32PM (#17852898)
That is a hyper parameter to the model. It is not learned.
Re:China Built the World’s Largest Telescope, Then Came the ... by EZ-E on Sunday August 26, 2018 @11:57PM (#17849349)
It's probably closer to statistics than to machine-learning at this stage. People just love to say they are doing machine learning, it's "sexy"
Re:China Built the World’s Largest Telescope, Then Came the ... by xevb3k on Monday August 27, 2018 @12:19AM (#17849407)
It just says “developing”. I would imagine this is code for “writing papers about”. I’m sure those papers use whatever technique is currently hot and will produce a decent publication. I doubt they use simple statistical approaches, because that’s likely not novel or publishable.

This is all fine. However, the problem is suggesting that this is a somehow reasonable/tenable solution to solving the noise issues...

Maybe the noise can be removed by other methods, maybe the noise doesn’t matter, but magic doesn’t solve the noise issues.

Re:China Built the World’s Largest Telescope, Then Came the ... by martinpw on Monday August 27, 2018 @01:37AM (#17849643)
> Maybe the noise can be removed by other methods, maybe the noise doesn’t matter, but magic doesn’t solve the noise issues.

The noise definitely matters. When you are observing with a telescope you are almost by definition working at the limit of the signal to noise ratio of the instrument, otherwise you would be asked to use a less powerful instrument (since the observing time on these larger instruments is so precious).

I worked on a project analyzing data from an X-ray telescope back in my research days. We spent 3 full years building an incredibly detailed model of the instrument noise, just so we could subtract it out and have confidence in our remaining signal. That was longer than the time we spent doing the actual science on the corrected data.

Re:China Built the World’s Largest Telescope, Then Came the ... by qubitcoder on Monday August 27, 2018 @06:26PM (#17855074)
For another perspective, check out this article in Symmetry Magazine, "Neural networks meet space" [1].

> "Analyses that typically take weeks to months to complete, that require the input of experts and that are computationally demanding, can be done by neural nets within a fraction of a second, in a fully automated way and, in principle, on a cell phone’s computer chip,” says postdoctoral fellow Laurence Perreault Levasseur, a co-author of a study published today in Nature."

> "The team at the Kavli Institute for Particle Astrophysics and Cosmology (KIPAC), a joint institute of SLAC and Stanford, used neural networks to analyze images of strong gravitational lensing, where the image of a faraway galaxy is multiplied and distorted into rings and arcs by the gravity of a massive object, such as a galaxy cluster, that’s closer to us. The distortions provide important clues about how mass is distributed in space and how that distribution changes over time – properties linked to invisible dark matter that makes up 85 percent of all matter in the universe and to dark energy that’s accelerating the expansion of the universe."

> "Until now this type of analysis has been a tedious process that involves comparing actual images of lenses with a large number of computer simulations of mathematical lensing models. This can take weeks to months for a single lens."

> "But with the neural networks, the researchers were able to do the same analysis in a few seconds, which they demonstrated using real images from NASA’s Hubble Space Telescope and simulated ones."

[1] https://www.symmetrymagazine.org/article/neural-networks-mee...

Re:China Built the World’s Largest Telescope, Then Came the ... by jasone on Monday August 27, 2018 @02:51AM (#17849850)
We had a morally similar concept to "machine learning" in academia during my grad school days: "non-parametric statistic". To me they were anathema, but non-parametric statistics are shockingly common both in academia and in industry. They work well, except when they don't.
Re:China Built the World’s Largest Telescope, Then Came the ... by MrLeap on Monday August 27, 2018 @06:22AM (#17850530)
non-parametrics! I took that class in college. This brings back memories. I got the impression you use non parametrics when you've got a sample size of 4 and you don't want all your statistics being un-legit sounding numbers like 25, 22.5, 50.. etc. "Increase" our sample size by adding every permutation of the 4 data points we started with to the sample! Gets you some cool statsi numbers like 63.1924.
Re:China Built the World’s Largest Telescope, Then Came the ... by krona on Monday August 27, 2018 @04:47AM (#17850235)
They work well, except when they don't.

Just like parametric statistics, then?

Re:China Built the World’s Largest Telescope, Then Came the ... by tabtab on Monday August 27, 2018 @02:40PM (#17853336)
I wonder if multiple smaller scopes electronically "connected" would be cheaper than one big one.