If you only work with fictitious data, the conclusions of your analysis and models... will be fictitious.
That doesn't have to mean it's worse though. Testing with 10,000,000 fictitious accounts generated to cover lots of permutations of user data is a great idea even if the real data won't ever have some of those permutations. It's a testing technique called "fuzzing", and it's quite common.
It entirely depends on what you're trying to deal with. You may solve some problems there but looking at real data is a must for solving others.
How do you ensure your fuzzed data has statistical signal that approximates those present in the true data?
That’s testing not modeling. If someone asks you to do analysis of a drug trial and you come back with a report based on fictitious user data because “devs should not have access to real data” you’ll be laughed out the room.
>If you only work with fictitious data, the conclusions of your analysis and models... will be fictitious.
Not necessarily, there is a lot of work done on synthetic data which closely matches the properties of the real data both in academia and increasingly in industry.
Re:Running VSCode in Docker
by marcus_holmes on Friday April 05, 2019 @07:42AM
(#19581564)
sure, but are data scientists analysing vast amounts of actual customer data really using VSCode?
Yes... What do you think they’d use? Those who prefer python are using PyCharm and VSCode, and then some use Jupyther, while others use Rstudio, matlab or Mathematica. Of cause you also have some doing emacs or vim, but that’s a minority.
Even anonymised date (which most healthcare data is btw.) has strict requirements on access control and copying.
As for “develop and test against”. There’s the entire field of data science and modeling that’s not focused with developing systems to expose the data, but doing analysis on the data.