https://medium.com/@rreisman/as-we-will-think-the-legacy-of-...
https://en.wikipedia.org/wiki/Provenance gives more on the term and the way it shows up.
More interestingly, both wine and art have had their provenance hinges widely exploited for massive profit while posh people think they're enjoying something exclusive.
course, I've also never heard of or touched the software you listed there either, but that may be because I don't view the data science and machine learning I'm interested in as being about specific software or vendor software...
sounds more database- lingo to me...
"Provenance" just means where the data came from. [1]
It's one of those shibboleths and terms of art used by people in industry. If you go to trade-shows you'll hear it being used -- it's worth knowing if nothing else but for its sociological value among the data software tools crowd.
Side: it's a little like the word "inference" being used as a verb by folks in AI (example usage: we use GPUs to speed up model "inferencing") -- in AI, to inference means to "predict". It's a term of art. If someone with a traditional statistics background went to a deep learning conference, they are likely to be very confused because in traditional statistics, inference means to obtain parameters θ in a model y = f(x,θ), whereas in AI, inferencing refers to obtaining y.
[1] https://en.wikipedia.org/wiki/Data_lineage#Data_provenance
Where I work, (a large pharmaceutical), these notebooks are taken very seriously by biologists, chemists, and chemical engineers, and increasingly are shaping the mindset of our data scientists (who have yet to adopt them).
Given the longstanding practice of documenting experiment design and method, I think it's probably long overdue that the exploratory analysis of experiment-based data must also adopt more rigorous governance to ensure that necessity and sufficiency are ensured when drawing inferences from experiment, especially when the data used was not originally intended to answer the current question posed.
A few publications from ~2011-2015 period:
http://ceur-ws.org/Vol-1558/paper37.pdf
https://ieeexplore.ieee.org/document/5739644
https://link.springer.com/chapter/10.1007/978-3-642-53974-9_...
Add a variety of additional links dating back a bit further (note the emphasis in this case on research data and tracking state of an experiment).
https://nnlm.gov/data/thesaurus/data-provenance
Data provenance is not a database / data warehouse term. It is uniquely and specifically a basic “101” concept of statistical science and ML / data science, where the custody and tracking of data are specifically tied to iterations of experiments, prototypes and research, for the sake of reproducibility.
If I was interviewing an experienced statistical researcher and they didn’t at least have a working knowledge of the core concepts, that would be a huge red flag.
Another poster mentioned vendor brochures and trade shows, which is in line with my expectations about which community it stems from, and also explains why I've never heard of it because I try to keep away from such environments these days.
Everywhere I've been the things which I take to make up "provenance" have generally been referred to under the simple label of "data quality", with separate subset definitions and measures such as timeliness, source, authority, format, history, suitability, verification, etc.
Of course, that's assuming people even worry about such things. In practice, let's be frank, anyone who's worked with data science knows they actually get shorter shrift than they deserve in practice: I'm probably among a minority of people in the real world who actually take things seriously, and I find myself on a constant crusade to remind people that just because a data point exists in a data set doesn't mean it's useful/ appropriate/ truthful/ unbiased.
data quality is a bit problematic, because I can see it being used by people who think provenance doesn't have any thing to do with quality, and from a variety of fields, but it is also infinitely more popular according to historical search trends, and in my last three jobs provenance would fall under the data quality framework.
In all of them, data provenance has been a first class consideration of machine learning and data platform teams, like a day-to-day concern and baked in to architecture review guidelines and production checklists and whatnot for every ML project.
In many of these companies we had teams of 20-40 ML scientists, all of whom knew about data provenance as a first class consideration in their work, had experience with it from their past jobs and academic programs, and considered it on equal footing with any aspect of data curation, model selection, model training and model serving.
Shrug, such is life I guess. That being said, I care deeply about this stuff (but didn't have a word), so perhaps it will be easier to convince people to pay attention to the data with said word.
I’ve worked as a data engineer for the last two years and never heard of this being used in this context before.
Typically the word “data lineage” is used to mean this in my experience.
I don’t think I’ve ever been in a meeting where someone mentioned provenance except referring to a show about paintings.
Lineage isnt the same thing, being a more specific technical term referring to keeping the history of datasets and where they came from (basically), but people actually say the words “data governance” and “lineage”.
edit: I should add that I'm definitely in favour of having provenance in ML systems, and libraries layered on top are the way that people currently do that. It's just odd that people aren't working on adding that support directly into scikit-learn/TF/pytorch etc.
I totally agree that where you have framework hooks, you should have provenance, but given there's no standard for what provenance is, no defacto open-source platform, the sklearn and tf and pytorch folks rightly steer clear. We see that if you have a shared file system, you can use conventions for path names (features go in 'featurestore', training data in 'training', models in 'models', etc), to capture a ton of provenance data.
Professor Margo Seltzer (https://www.seltzer.com/margo/) is a well-known researcher in the area of provenance. I highly recommend reading her papers if you're interested, starting with her USENIX ATC 2006 paper "Provenance-Aware Storage Systems".
But the money is in replacing humans.
We had built it precisely to free us from certain repetitive things in machine learning projects [environment set up, near real-time collaboration on notebooks, scheduling long-running notebooks, experiment tracking, model deployment and monitoring]. We used to scramble and do all that, request help from our colleagues and pull them from what they were doing. This was really taxing and bad for morale, jumping around from one context to another.
I had a huge smile contemplating all the work I was about to not do.
There are many things where the humans themselves ought to be "augmented". Case in point, in some projects involving predictive maintenance, the stakes of an incident can be around $100MM and all these processes depend on a human being alert at all times during their very long shift, with a bunch of other things happening simultaneously. This is very stressful and these people actually want to be "augmented". They want something to help them and catch things they would have missed because they haven't had proper sleep or were too busy solving another urgent and important problem. It is the people themselves who come to us and ask us for our help to help them solve these problems.
It may sound cheeky, and in many cases at many companies it is cheeky and it is PR like saying "partners" instead of "drivers", or "dashers" instead of "delivery person". In some cases it really is what happens. At least from my biased perspective with the actual humans who were asking for "augmentation" to do their job.
Any time a tool evolves, it changes not only how a task is done but also why. In the case of implementing a business process, the revision process is best served by reconsidering why what is done now and taking the opportunity to evolve the old role into making a richer contribution that introduces a new and improved path through the problem space.
That's IA, and IMO, it's the Great White Hope that AI might yet lead to a future world that engages humans more rather than the default dystopia where we're all redundant and irrelevant.
I'm not a data scientist and I've never encountered that term "provenance" before but I've encountered the problem he talks about in the wild here and there and have searched for a good way to describe it. His ultrasound example is a great, chilling, example of it.
I also like the term "Intelligence Augmentation" (IA). I've worked for a couple companies who liberally sprinkled the term AI in their marketing content. I always rolled my eyes when I came across it or it came up in say a job interview. What we were really doing, more practically and valuably, was this: IA through II (Intelligent Infrastructure), where the Intelligent Infrastructure was little more than a web view on a database that was previously obscured or somewhat arbitrarily constrained to one or two users.