You can use PSI for that. And it actually makes more sense when all applications listen on the global PSI indicators so that if one application would start thrashing another can free up memory cooperatively. But if needed it can also be done per cgroup.