even within the same type of tissue, different patients are going to have different mutations/subclones in their tumors.

Depends on if that mechanism of action works for the particular exact set of mutations that the given tumor has (every cancer is essentially unique). Part of the success of the work in that paper is identifying specific markers that indicate that the drug is going to work in that tumor

Both dostarlimab and pembro are PD-1 blockers, yet they are labeled for two different cancers. If all tumors have different characteristics, why are these not labeled for all cancers, in general, that display these expressions? Is this part of the game that the FDA plays?

They are:

https://www.fda.gov/news-events/press-announcements/fda-appr...

https://en.wikipedia.org/wiki/Tissue-agnostic_cancer_drug

If by "game" you mean rigorous regulatory process to ensure safety and efficacy, then yeah. Different monoclonal antibodies even with the same target can behave differently. You'd have to specifically design a trial and seek approval for an indication like the one you described. Which people have done and are doing. It's an exciting field of research!

different cell types express different genes at different amounts at different times

each tumor cell/ subclone has different mutations at different times