cmonsen

Got around to reading the primary source. I am surprised there isn't more skepticism from this group, but it may have to do with not having time to read the paper. A few reasons to be skeptical of the results (some are stronger than others):

1) First, required reading for anyone interpreting medical studies: http://jama.jamanetwork.com/article.aspx?.articleid=201218 TLDR; is that ~40% of _randomized_ (generally considered highest level of quality) studies and maybe more are proven wrong by subsequent ones resulting from publication bias. Consider that this would not have been published in a high profile journal if it showed no effect, hence pub bias. In other words, we're not seeing the similarly large analyses that show no effect because they're not published.

2) The adjusted analyses have confidence intervals on the odds ratio that overlap with 1.0 except for the highest dose group. There is evidence of a dose response relationship, but there is a problem in how they measure dose response. Cumulative dose biases towards people who have been on medications for a long time. People who are able to take medications (ie are older) for a long time are at greater risk for dementia.

3) It is problematic that they assume any effect on their endpoint is a class effect (ie true of all anticholinergics). The relationship with one medication may be driving this effect entirely. I would like to see, for example, oxybutynin pulled out of the analysis which is never prescribed for allergies and more often for neurological conditions, which could be an unmeasured confounder depending on how they modeled the comorbidities.

4) Controlled analyses are difficult to do well. Most of the time it controls for linear effects, meaning synergistic effects (which are probably involved in development of dementia) are not captured unless explicitly modeled. I do not see that they did this here.

5) I am further suspicious of the way they obtained fill data. As you are noting, many OTC anticholinergics are not "dispensed" in a way typical of prescription anti-cholinergics. If I bought Benadryl at CVS or even if I bought the pre-packaged bottle at the Group Health pharmacy, it's pretty clear to me that neither of these would be reflected in the data. This skews heavily toward prescription anti-cholinergics, which tend to be given for psych/neuro indications which themselves have an association (though likely not a causal one) with dementia. Again, unclear if they modeled all of these.

Overall, I share the sentiment that medical literature needs to be more open. I think it's unfortunate that current state makes it hard to confirm analyses like these with a great degree of certainty.

Completely agree with reproducibility of research. Would be great to further improve such algorithms using data collected from the public--selection issues and inaccuracies aside.

The algorithm is in the appendix of the article the tool links to.

Input fields are limited via javascript so may not have worked if you have it disabled. You're right we should do it server side as well.

Thanks for the feedback.

Glad you found it useful. Stay tuned for similar tools.

Definitely. But with still mostly paper recording, limited data standards, and particularly onerous privacy requirements (justifiably so), it is not a foregone conclusion that the big data trend will impact medicine anytime soon.

Privacy policy posted. It's in the footer of symcat.com. Thanks for your patience and let us know if you have any feedback!

We observed that we had exact concept matches for over 2/3rds of the symptoms searched on the first try.

To get the rest we are doing two things: 1. adding common queries without matches into our symptom vocab 2. improving the suggested searches algorithm

True, but if that alone were the explanation then "shortness of breath, vomiting, and chest pain" would also be higher.

That's true, for HIPAA compliance, there is no negotiation and we will meet that standard.

But there inevitably will be some user concerns that fall outside of HIPAA compliance. So, we see HIPAA+HITECH as a minimum requirement. We don't expect it to be sufficient, however, and that's where user feedback, the "user ethics panel" if you will, comes in.

Apologies for the oversight. Will push it out to the site today.

I'm glad you made that point. We are very aware of how important user privacy is for this sensitive information. We want to begin the conversation with users while we're young so that we don't mess it up when we're big (a la facebook).

As a general principal, we will only reflect data back that has been fully anonymized. In fact, we don't collect personally identifiable information (that's why there are only 3 choices for age right now). We are building HIPAA-compliant software (even though it's not legally applicable to us yet). We have a team of advisors, including privacy experts, but honestly, we believe the best ethics panel will come from the users and are very interested in feedback in this respect.

We were getting a lot of traffic in a short amount of time that day, so the site performance was lacking. You might have better luck trying again (maybe wait a few hours until this new HN traffic dies down).

I welcome the opinion of a legal expertise on your later point, but we're taking the appropriate measures based on the advice we've received.

Hoo boy, thanks for catching that! Was wondering why we weren't getting any follows. Fixed now.

I didn't know that. I have to get out (watch TV?) more. Robot battle it is, then.

Should be fixed now. Thanks!

Agreed, and that's a very reasonable place to start for conditions that are not challenging diagnoses but rather explanations for common symptoms (eg too little sleep, bumped my knee, etc).

WRT abuse, we're going to require the log-in and user consent before collecting any data and allow people to maintain, in effect, their own medical record. We can monitor for abuse (eg someone who seems to visit a hospital thousands of times / day).

Thanks for the suggestions! We are fortunate to have some great advisors including practicing physicians, medical informatics specialists, and health care entrepreneurs. We're part of Blueprint Health, a TechStars-affiliated health startup accelerator.

You point to a challenge for any patient-facing patient decision aid -- getting the language right. We are building on the work of the U of Utah and building a robust consumer health vocabulary based on query data.

I'm sorry if it is abusive. That was not my intention. It is not unusual for founders to post to HN.

We'd love to incorporate a DXPlain or QMR as a starting point, but the true value of this is in learning from additional use/data.

The evidence suggests that 15% of medical diagnoses are incorrect. Being 2 months away from finishing my MD, I have a sense that this is not because doctors are failing to do their job, but rather, because medical diagnosis is hard.

Most people believe that their doctor gives them a diagnosis and are 100% certain of it, but in reality medicine is a highly probabilistic domain. There's a great article about a family's story with misdiagnosis in their child (http://www.slate.com/articles/double_x/doublex/2011/07/how_f...). The diagnosis was initially strep, but it soon became evident that the diagnosis was the much more rare Kawasaki disease. Symcat is an attempt to let patients know about both (ie "This is probably strep, but you should continue to monitor because Kawasaki disease can also cause this and the only way you know is if your symptoms persist").

We would love to get more docs on board and we've been working with physicians at Hopkins to make this as useful to patients as possible.

Completely agree that this lives on the quality of the data. We would like for users to contribute their data (a la patientslikeme). We also are exploring partnerships with health care providers and data stores for getting additional data.

Thanks!

Honestly, Google. We need to make sure that it is easily indexed and understood by crawlers. Some med students have found it useful though.

Will fix it, thanks!

Haha. My co-founder and I went back and forth about the right message for our HN title. I completely agree with you but he won the argument. Does anyone know of a good A/B testing site for HN titles? ;)