This was one of the original arguments in favor of copyright, except in reverse, as the poverty of heirs was an argument in favor of copyright:
When, in 1749, Thomas Newton's (1704-1782) edition of Paradise Lost was published, it carried details about the impoverished circumstances in which Elizabeth Foster, Milton's granddaughter, lived.[33] The account gave rise to a performance of Milton's Comus at the Theatre Royal, Drury Lane, from which Elizabeth received £130, the profits of the evening. When aligned with the fact that Milton himself received only £20 from Simmons, the tale became a staple in subsequent accounts of neglected genius and avaricious publishers.
Deazley, R. (2008) ‘Commentary on Milton's Contract 1667', in Primary Sources on Copyright (1450-1900), eds L. Bently & M. Kretschmer, www.copyrighthistory.org
Still that cuts in both directions, in that you could use it to argue that other properties don't necessarily need to be heritable.
Either way it's pretty clear the "limited times" they were imagining were a portion of a lifespan, not multiple lifespans. It's pretty ridiculous that as the world moves faster these durations are increasing rather than decreasing.
"To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;"
The question is does the heritability of IP promote the progress of science and useful arts, or does it simply enrich heirs? I can see the argument that people wouldn't create art or do science if they couldn't profit from it, but it's much greyer whether they wouldn't do it if they couldn't pass it to their heirs.