I think a reasonable default would be for the customer's account to continue to reference the private material, but to disallow private access to it.

Then a customer could:

1. Convert it to public and access it as such.

2. Remove it.

3. Leave it alone, possibly intending to privately access it in the future if they can re-subscribe. (It'd seem reasonable to me if a service provider opted to automatically delete inaccessible content after some time-out period of non-subscription; I'd see their retention of such inaccessible content as a professional courtesy rather than an obligation of theirs unless the contract says otherwise.)

Then I'd think that a good service would offer migration tools to help make these options easier on the customer. For example, offer an easy option to make everything public or delete everything.

But it's hard for me to see how forcing publication upon canceling a service could be defensible without some sort of contractual provision to that effect.

Sure but one you fix in your product by having a grace period for accounts that are closing. The other you can't fix because once something on the internet is public you can't undo it.