News for hackers. Stuff that matters.

Ask HN: Ways to automatically make inferences from data?

Suppose I have sales data - it is easy to run a few queries to find out top selling items, top regions etc.

But, is it possible to do this on any set of data, without knowing anything meta about the data at all? In other words, can this be generalized? Are there any models, theories I can learn to achieve this?

( 6 comments | 3 points | on HN )
Re:Ask HN: Ways to automatically make inferences from data? by JPLeRouzic on Sunday May 19, 2019 @05:03PM (#19955509)
I am not a specialist of this field but here is how I would deal with this problem:

My understanding is that it is possible to find the main components (items, regions, etc) in a dataset, for example with PCA [0]. However it will not name those components, but it might be quite easy to infer the name of each component. Once you know the data in one component, you can find their min/max. I guess there are several similar mathematical techniques to do the same job and there are also several user friendly Business Intelligence software.

[0] https://en.wikipedia.org/wiki/Principal_component_analysis

Re:Ask HN: Ways to automatically make inferences from data? by akudha on Sunday May 19, 2019 @05:12PM (#19955549)
If I had a CSV, I can import it into a database and simply run queries, systematically, isn't it?
Re:Ask HN: Ways to automatically make inferences from data? by JPLeRouzic on Monday May 20, 2019 @03:28PM (#19964081)
You can alway do that, the question is (as the other comments told) "will your users get meaningful answers?"

You could simply use Excel (it has PCA) also to test your ideas. There are many resources on Internet.

Re:Ask HN: Ways to automatically make inferences from data? by thedevindevops on Monday May 20, 2019 @03:25AM (#19958194)
Yes - but not in the way you think.

A program can be written to read in a dataset and correlate each set of data with every other set of data in that dataset (think producing a set of graphs comparing every combination of 2 properties of that dataset).

Somewhere in that mess of graphs there will be useful ones but the program won't be able to tell the difference.

You still need a human for that.

Re:Ask HN: Ways to automatically make inferences from data? by dhkxh on Monday May 20, 2019 @04:25AM (#19958421)
Your title is about inferences, but your text describes a summary, a descriptive statistic or an aggregation - they are very different problems. It's quite straightforward to "find a top item" regardless of the dataset but I don't know why you would want this automated at all.
Re:Ask HN: Ways to automatically make inferences from data? by natalyarostova on Monday May 20, 2019 @03:14AM (#19958155)
As a general answer to a general question: not really.

Domain knowledge and prior knowledge on how the data is associated with reality exists in the brain of the human, who expresses that using code and science for the problem at hand.

Otherwise the computer doesn't know the difference between sales data. And any other data.