I haven't used Privoxy for ad blocking but that's a novel idea.

Just wanted to comment that setting up your own CA is the solution I chose to go with to continue filtering HTTPS sites. Certificate pinning can prevent this, but apps or devices that employ that can simply be uninstalled or resold.

The move to HTTPS is intended to benefit consumers, it doesn't have to be an obstacle to viewing sites the way you want to.

Your solution to an application using HSTS isn't just to punch it straight through the proxy, but to sell the device?

Yes.

I believe in being able to verify that a device or app isn't leaking sensitive information, and I enforce that using HTTPS interception. As an app developer if you attempt to lock me out of the communications leaving my network, I the choice of potentially compromising my security and privacy or the choice of blocking the traffic, and I choose to take a hard line when it comes to security.

The frustrating thing is that HTTPS is typically seen as good guys (server operator) vs bad guys (anything who isn't the client browser). But there's a lot of gray areas.

Take any network that wants to scan HTTPS traffic for incoming viruses at the perimeter for example, which is a lot of corporate networks. Any use of certificate pinning restricts the network owner's ability to virus-scan or apply Data Leakage Prevention rules to that traffic.

We probably both agree that virus scanners are unlikely to catch emerging threats and that DLP rules are easily bypassed, but they are also layers of a much larger security onion.

Edit: HSTS headers can be stripped in transit, but certificate pinning requires significantly more effort to defeat and IMO isn't worth the effort, that's why I talked about cert pinning.

I don't think you're going to catch a lot of what you think you're going to catch with HTTP inspection. Actually getting data out of a compromised system could happen with all manor of seemingly innocent information that would pass through even a fine tooth comb of every packet. What if there's malware leaking data by appending whitespace on the end of URLs, messing with the timing of DNS requests, adding pixels to images on the fly? I don't think it's humanly possible to validate the amount of data which floods out of systems on a daily basis.

I agree, it's easy to get around these systems with minimal effort.

I disagree that I shouldn't try to catch the low hanging fruit because of the existence of higher hanging fruit.

You should let me as a consumer or network operator make a choice about whether pinning is mandatory or not.

Like it or not, some people have a legit need to protect the data on their network.

Being able to look at data inside an SSL tunnel won't tell you if it's communicating your stuff out or not. Only the most obvious leaks are going to be caught that way.

Do you realize that you disagree with your first sentence in your second sentence?

We understand that only the most obvious leaks will be caught this way. The existence of more sophisticated attackers shouldn't discourage you from trying to catch the less sophisticated ones.

I hypothesize the most reliably useful information is the metadata concerning each flow itself: what device is on the local end, what IP's on the other end, with what public key and cert chain in the KEX, at what time of day, and how much data is being transferred.

If you really want to see the data transferred by a specific app, inspect/modify your app's source code. If you don't have the app's source code, and you are worried about what data it transmits, what are you doing using it?

It's not a contradiction, though it could have been worded better. You're not ever going to be able to tell with certainty that data is not being leaked. As a person with some sensitive files, does it matter how sophisticated the attacker was? The end result is the same either way if they get out.

Sure it can. Ever heard of DLP?

Most leaks aren't terribly complicated.

When you MITM SSL like this you severely undermine the security of every application speaking HTTPS.

I disagree because the only people that will be impacted by me MITM-ing SSL connections at my home is me or people who have intruded into my network, and on a corporate network the only exposure is to employees who have agreed to corporate policies (some sites like banks are not accessible through the corporate filter for employee privacy reasons).

If I was an ISP or network provider I would have to agree with you.