Exactly. The PDF's work. They won't break. You can see all the information with your own eyes. You can send them by e-mail.

A wizard-type system hides most of the information from you, it might have bugs you aren't aware of, if you want to glance at an alternative path you can't, it's going to be locked into registered users, the system can go down.

I think much more intelligent computer systems are the future in health care, but I doubt the way to start is with yet another custom tool designed specifically for cancer guidelines and nothing else.

> it's going to be locked into registered users, the system can go down

I didn't see anything in the screenshots presented that wouldn't be doable in a single HTML file containing the data, styles and scripts?

This is a countercultural idea but it fits so many use cases; it's a tragedy we don't do this more often. The two options are either PDF or SaaS.

> The PDF's work. They won't break.

Not just that, PDFs are one of the few formats, where i'm willing to bet my own money, that they'll still work in 10 or 20 years.

Even basic html has changed, layouts look different depending on many factors, and even the <blink>-ing doesn't work anymore.

A case specific PDF could be created and stored in the patient's electronic records. Such PDF could just highlight the decision three path.

Sure, PDF/A is an ISO-standardized subset of the larger PDF spec designed expressly for archival purposes. You could do that with HTML but then how would you get your crypto mining AI chat bot powered by WASM to work?

The author is proposing that the DAG representation be in addition to the PDF:

>The organizations drafting guidelines should release them in structured, machine-interpretable formats in addition to the downloadable PDFs.

My opinion: Ideally the PDF could be generated from the underlying DAG -- that would give you confidence that everything in the PDF has been captured in the DAG.

You could generate the document from the graph and then attach it as data.

> could generate the document from the graph and then attach it as data

Much easier for doctors to draft PDFs than graphs.

I have not drafted a PDF myself and I doubt doctors will. They work in a text writer or spreadsheet application and then export or print to PDF would be my guess. An interactive interface could spit out the PDF with the decision three in the end. This solution would still mean the decision tree source is in some software package.

Hey author here! Appreciate the feedback! Agreed on importance of portability and durability.

I'm not trying to build this out or sell it as a tool to providers. Just wanted to demo what you could do with structured guidelines. I don't think there's any reason this would have to be unique to a practice or emr.

As sister comments mentioned, I think the ideal case here would be if the guideline institutions released the structured representations of the guidelines along with the PDF versions. They could use a tool to draft them that could export in both formats. Oncologists could use the PDFs still, and systems could lean into the structured data.

The cancer reporting protocols from the College of American Pathologists are available in structured format (1). No major laboratory information system vendor properly implements them, properly, and their implementation errors cause some not-insignificant problems with patient care (oncologists calling the lab asking for clarification, etc). This has pushed labs to make policies disallowing the use of those modules and individual pathologists reverting to their own non-portable templates in Word documents.

The medical information systems vendors are right up there with health insurance companies in terms of their investment in ensuring patient deaths. Ensuring. With an E.

(1) https://www.cap.org/protocols-and-guidelines/electronic-canc...

> The medical information systems vendors are right up there with health insurance companies in terms of their investment in ensuring patient deaths. Ensuring. With an E.

Can you expand on this?

Medical information system vendors only care about making a profit, not implementing actual solutions. The discrepancies between systems can lead to bad information which can cost people their life.

As an analogy, imagine if the consequence of Oracle doing Oracle-as-usual things was worse medical outcomes. But they did them anyway for profit.

That's basically medical information system vendors.

The fact that the US hasn't pushed open source EMRs through CMS is insane. It's literally the perfect problem for an open solution.

It's worse than that. VistA is a world-class open source EMR that the VA has been trying to kill for decades.

VistA was useful in it's time but it's hardly world class anymore. There were fundamental problems with the platform stack and data model which made it effectively impossible to keep moving forward.

I love open source EMRs, but has any major country adopted open source EMRs?

I know OpenMRS exists but is mainly used within developing nations.

The US has Vista, made by VA, and it is a beast and no one really wants to use it.

If I understand correctly, Estonia made their own EMR/EHR from scratch. The government produced (and commissioned?) software is all open source. https://koodivaramu.eesti.ee/explore

EMR software seems like something that shouldn't be that hard. It's fundamentally a CRUD. Sure, there's a lot of legacy to interface with, but medical software seems like a deeply dysfunctional and probably corrupt industry.

It’s a famous “should be easy” use case. I think this is wrong only because no one does it.

I'm sure there's a lot of work, but hundreds of millions per deployment is not justifiable. The Finnish EPIC deployment has cost almost a billion euros.

Estonia's from-scratch system was reportedly about 10 million euros.

It wouldn't be appropriate for the federal government to push any particular product. They have certified open source EHRs. It's not at all clear that increased adoption of those would improve patient outcomes.

https://chpl.healthit.gov/#/search

Since Oracle bought Cerner a few years ago, no imagination needed. Sadly, since Cerner has lots of good people who want to make good products.

>The fact that the US hasn't pushed open source EMRs through CMS is insane. It's literally the perfect problem for an open solution.

It's not insane, it's because the US is an oligarchy. And it's about to go even more oligarchy on steroids in the next year.

What explains most other democracies not doing it?

Is Sweden an oligarchy, too? Or France? Etc etc

I mean, you're attributing malice, but it could just be that reliably implementing the formats is a really really hard problem?

How about fixing the format? Something that is obviously broken and resulting in patient deaths should really be considered a top priority. It's either malice or masskve incompetence. If these protocols were open there would definitely be volunteers willing to help fix it.

Incompetence at this level is intentional, it means someone doesn't think they'll see RoI from investing resources into improving it. Calling it malice is appropriate I feel.

If there is no ROI, investing further resources would be charity work. I don’t think it’s accurate to call a company not doing so malicious.

Not actively malicious perhaps, but prioritising profits over lives is evil. Either you take care to make sure the systems you sell lead to the best possible outcomes, or you get out of the sector.

The company not existing at all might be worse though? I think it’s too easy to make blanket judgments like that from the outside, and it would be the job of regulation to counteract adverse incentives in the field.

Agree that most companies prioritize profits over lives in an unconscionable manner, but there's a point of diminishing returns where eventually you can save a few more lives, but at an astronomical cost. Auto manufacturers have the same dilemma: spend a few hundred million dollars adding safety features, or nix the features and hope to lose less than that in lawsuits?

Eventually the question will be, how far do we really need to go, i.e. how much profit do we allow ourselves before it's morally untenable and we should plow it back into R&D? Unfortunately, as long as health care is for-profit, and absent effective regulation, companies will always err on the side of profit.

You seem to think that the default assumption is that fixing the format is easy/feasible, and I don't see why. Do you have domain knowledge pointing that way?

It's a truism in machine learning that curating and massaging your dataset is the most labor-intensive and error-prone part of any project. I don't why that would stop being true in healthcare just because lives are on the line.

I think there are more options than malice or incompetence. My theory is difficulty.

There’s multiple countries with socialized medicine and no profit motive and it’s still not solved.

I think it’s just really complex with high negative consequences from a mistake. It takes lots of investment with good coordination to solve and there’s an “easy workaround” with pdfs that distributes liability to practitioners.

Healthcare suffers from strict regulatory requirements, underinvestment in organic IT capabilities, and huge integration challenges (system-to-system).

Layering any sort of data standard into that environment (and evolving it in a timely manner!) is nigh impossible without an external impetus forcing action (read: government payer mandate).

You're making a lot of unsupported assumptions. There's no reliable evidence that this is causing patient deaths, or that a different format would reduce the death rate.

It doesn't look like the XML data is freely accessible.

If I could get access to this data as a random student on the internet, I'd love to create an open source tool that generates an interactive visualization.

The problem is that a bug could kill people.

People could potentially properly implement them if they were open and available:

"Contact the CAP for more information about licensing and using the CAP electronic Cancer Protocols for cancer reporting at your institution."

This stinks of the same gate-keeping that places like NIST and ISO do, charging you for access to their "standards".

For liability reasons alone, you cannot just have random people working on health/lab stuff and the requisite vendors have access to these standards.

According to what killjoywashere said, the vendors do not want to implement these standards. So if CAP wants the standards to be relevant, they should release them for random people to implement.

Aren’t all NIST standards free as they are a government body?

>Agreed on importance of portability and durability.

I think "importance" is understating it, because permanent consistency is practically the only reason we all (still) use PDFs in quite literally every professional environment as a lowest common denominator industrial standard.

PDFs will always render the same, whether on paper or a screen of any size connected to a computer of any configuration. PDFs will almost always open and work given Adobe Reader, which these days is simply embedded in Chrome.

PDFs will almost certainly Just Work(tm), and Just Working(tm) is a god damn virtue in the professional world because time is money and nobody wants to be embarrassed handing out unusable documents.

PDFs generally will look close enough to the original intent that they will almost always be usable, but will not always render the same. If nothing else, there are seemingly endless font issues.

In this day and age that seems increasingly like a solved problem to most end users, often a client-side issue or using a very old method of generating a PDF?

Modern PDF supports font embedding of various kinds (legality is left as an exercise to the PDF author) and supports 14 standard font faces which can be specified for compatibility, though more often document authors probably assume a system font is available or embed one.

There are still problems with the format as it foremost focuses on document display rather than document structure or intent, and accessibility support in documents is often rare to non-existent outside of government use cases or maybe Word and the like.

A lot of usability improvements come from clients that make an attempt to parse the PDF to make the format appear smarter. macOS Preview can figure out where columns begin and end for natural text selection, Acrobat routinely generates an accessible version of a document after opening it, including some table detection. Honestly creative interpretation of PDF documents is possibly one of the best use cases of AI that I’ve ever heard of.

While a lot about PDF has changed over the years the basic standard was created to optimize for printing. It’s as if we started with GIF and added support to build interactive websites from GIFs. At its core, a PDF is just a representation of shapes on a page, and we added metadata that would hopefully identify glyphs, accessible alternative content, and smarter text/line selection, but it can fall apart if the PDF author is careless, malicious or didn’t expect certain content. It probably inherits all the weirdness of Unicode and then some, for example.

I would assume these decision tree PDF use a commonly available font. Layout and interpreted outcomes should be the same.

I believe you have good intentions, but someone would need to build it out and sell it. And it requires lots of maintenance. It’s too boring for an open source community.

There’s a whole industry that attempts to do what you do and there’s a reason why protocols keep getting punted back to pdf.

I agree it would be great to release structured representations. But I don’t think there’s a standard for that representation, so it’s kind of tricky as who will develop and maintain the data standard.

I worked on a decision support protocol for Ebola and it was really hard to get code sets released in Excel. Not to mention the actual decision gates in a way that is computable.

I hope we make progress on this, but I think the incentives are off for the work to make the data structures necessary.

You say this, but on the other hand, the author alleges that the places that use these custom tools achieve better outcomes. You didn't address this point one way or the other.

Do you think this is a completely fabricated non-explanation? It's not like the link says "the worst places use these custom tools."

Totally valid concerns. If you have time, I would like to show you my solution to get your thoughts as I believe I have found ways to mitigate all of your concerns. Currently I am using STCC (Schmitt-Thompson Clinical Content). I Have sent you some of the PDF's we use for testing.

It would, I imagine, be much easier to generate a PDF from the tool's internal flowchart representation than the other way around.

I think there’s value if it can scale down.

Community oncologists have limited technology resources as compared to a national cancer center. If we can make their lives easier, it can only be a good thing.

That said, I like published documents like PDFs - systems usually make it hard to conii ok are the June release from the September release.

I agree. However, since the PDF format supports structured data, one could in principle have it both ways, within a single file.

^ This. See, e.g., https://lab6.com/ for some interesting tricks with the PDF format.