News for hackers. Stuff that matters.

louden

Yes. Bishops in the US closed parishes in March. The Pope granted a dispensation so that Catholics do not have the obligation to attend Mass during Lent and Holy Week.
Re:D for Data Science: Calling R from D on Monday January 27, 2020 @12:06PM (#22161291)
One use case I have is embedding statistical calculations in larger applications. Calling R from the language the application is written in is a lot better than rewriting the statistical calculation in that language, especially when it is a complex model.
You are comparing a program where the benefits vastly outweigh the cost to a program where the benefit is mostly speculative and the cost known.

The cost of full body imaging is not just in the scan itself but in the procedures that follow it that require anasthesia (non-zero chance of killing the patient), cause pain and may cause secondary infection. In fact, just being in the hospital or doctors office to get the procedure has a non-zero risk to due to the concentration of sick people there.

People have commited suicide because of medical test results that turned out to be wrong. A decision has to be made about how many tests are performed and under what conditions. Unfortunately, that will cause harm to people no matter where we, as a society, draw that line.
In medicine, there are physically harmless tests that can still cause harm. Full body scans are one of them. If something is found, it will likely have an emotional impact on the patient. Not all people are equally rational. Not all people have a good grasp of the statistical nature of the findings. That applies to educated people and medical professionals who deal with this stuff everyday.

My point was not that either side was correct but rather that a simple statement like "more data is always better" is not as simple as they seem to think it is. Those of us who work in this field deal with balancing the risks as best we can so that we can detect the marginal elevated risk of something in a way so that the benefit outweighs the costs (financial, physical and emotional).

More data is not always better. The collection process itself can be dangerous or unpleasant for the patient.

Several years ago I worked on a study looking at the spread of bacterial lung infections in long term hospital patient. In order to get the data, we would have to intubate patients that did not need it. In this case, the people who approve these studies rightfully ruled that getting that data would be too painful for the patients to allow.

Biopsies caused by benign findings from a full body scan are not without pain, emotional distress, risk or cost to the patient. Period. It must be weighed against the benefit that the patient would recieve (all in the aggregate, or course).

Re:Common statistical tests are linear models on Thursday March 28, 2019 @04:35PM (#19516322)
When I was learning undergraduate mathematical statistics, my professor made it a point to connect the links between linear models and many different tests.

In the back of Statistical Inference by Casella and Berger, there is a great chart making similar connections between the most common statistical distributions.

There is a lot of rhyming in statistics.

Ask HN: What do you use to ensure your data is correct?

At what point in your data pipeline do you check that your data adhears to constraints on it? Is it embedded in the code that performs the ingestion, at the point of the analysis or, prehaps, do you have a utility that allows you to view data in your database that do not conform to constraints on it?

( Read More... | no comments | 3 points | on HN )
Re:Future of Statistical Programming on Saturday October 27, 2018 @02:38PM (#18317055)
The beauty of R (and Python) is the ecosystem, not necessarily the language. Most statisticians want libraries that they can use to do the modeling so that they can focus their time on the data, not the programming.
Re:What I Would Like to See in a Type System on Friday October 26, 2018 @09:17AM (#18308859)
I imagined that would be the case, but don't have the CS background to prove one way or the other. I am currently learning OCaml and would love to use it more, but my job keeps me tied to R, SAS and Python.
Re:Autonomous Trucks and the Future of the American Trucker on Thursday September 27, 2018 @05:58PM (#18089155)
It is much more than a lack of maintenance. The weight of the cargo, the temperature of the road, the way the cargo is distributed in/on the trailer and manufacture defects can all contribute to blowouts of servicable tires. Those issues also make the lifespan of identical tires vary by thousands of miles traveled.

While replacing tires more frequently would reduce the number of blowouts, it would be very wasteful. In the US, DOT inspections are frequent and they look at the tires to make sure they are in good condition.

Re:Using Deep Learning to Help Pathologists Find Tumors on Thursday June 21, 2018 @08:59AM (#17364638)
It would be nice to see the sensitivity and specificity of the technique and for humans. False positives and false negatives are not equal in medicine, so we should report in such a way that people can evaluate them.

In this type of cancer, a lower specificity is an acceptable trade off for a very high sensitivity.

Her consent would also be needed.
Even when we have whatever technology would make heathcare less limited, there will need to be ways to measure the prognosis of the patient. Otherwise we would be giving very unpleasant treatments to patients who only need palliative care (like chemo for terminal cancer patients).
> It's great but with healthcare I don't think people should be treated as dollar signs.

These kinds of models are also great for triage. Healthcare is a limited resource, especially in trauma situations which have been using models to measure survival for decades.

Re:America has a truck driver shortage on Tuesday May 29, 2018 @10:06AM (#17179254)
It is the same in the US. All trucks that transport across state lines has to have an electronic log. Tickets issued to the driver can be noted on the saftey record of the company and if there are enough violations, especially of certain types, the companies authority to transport can be revoked.
Re:America has a truck driver shortage on Tuesday May 29, 2018 @10:00AM (#17179202)
Truckers need $750k of liability insurance (but usually have $1M). For individual drivers, they are required to have liability, but it varies by state (and in some places a lot of people flaunt the law).

Ask HN: What does your project management tools look like for data science?

I have worked in several environments as a statistician/data scientist where the project management for my projects ranged from 'let me know when your done' to burndown charts and Agile. This got me thinking: what project management structure do other data scientist/statisticians use and is there a best set of practicies? For me, at the very least, I would like to see source control...

( Read More... | no comments | 2 points | on HN )
Re:Science’s Pirate Queen on Friday February 09, 2018 @11:19AM (#16341402)
> For instance, in 2013, the Obama administration mandated that copies of research conducted through federal agencies must be uploaded to free repositories within 12 months of publishing.

All federally funded research in the US is mandated to be open access.

StudyDesign.io provides a single place where statistical calculations for the design of experiments, such as sample size calculations and subject randomization, can be performed.
Re:Ask HN: Freelancer? Seeking freelancer? (July 2017) on Monday July 03, 2017 @12:51PM (#14689658)
SEEKING WORK - REMOTE (US Based)

As a data scientist with eight years of experience, I can take raw data, derive insights from them and turn them in to/embed them in production grade applications for business use. My goal with any project is to take your data and, with the help of statistics and machine learning, turn it into actionable insights. I look forward to discussing how I can help you.

Technologies:

AWS, Hadoop, Python, R and SAS.

Email:

[email protected]

Links:

https://github.com/louden

http://loudenanalytics.com

The methodology they used seems very good and the fact that they share the code is excellent.

However, in a study like this, data quality is likely going to be a problem. Collecting good data about the shootings themselves is incredibly difficult considering we have 18,000+ police jurisdictions in the United States and even mundane things like reporting can very between them. Furthermore, the chaos of the situation may make data quality suffer.

Unfortunately, the best we can do is announce the limitations of the studies, as the authors did.

Hispanics were in the study (see the Research Objectives) and have similar results to blacks. They just were not called out in the abstract.
Re:We are ruthless on code reviews on Wednesday April 27, 2016 @12:10PM (#11581714)
'We are rigorous on code reviews' would be a better approach. We don't have to be jerks to make sure that code is up to standards.
I have been on the other side of the table and I'm sure I came off as indifferent. The problem is, I have been thrown into the interview as an interviewer multiple times with about 1 hour notice when I have many other things to focus on.

It is definitely a broken process in many companies.

Re:Ask HNs: Who would you hire? on Tuesday April 26, 2016 @03:39PM (#11574911)
Candidate 1.

Having a person who is hard to work with can be demoralizing to the team he is working with which will lead others to loose productivity. Nobody is good enough to justify making the work environment toxic.

Older »