1. Is the app breaking or failing or not serving its purpose in any way? First find the problem, then look for the causes.
2. Let's say your app is using 29G of the memory in a 32GB instance. If you put it in a 64GB instance, will it use 29GB of that, 63GB of that, or does it continue to grow until it runs out of memory and restarts? The first case means you have correctly sized your instance. The second case means the app is good about using almost but not quite all the memory available. The third case is a problem.
So it seems it is the second case.
Memory, however is seldom the bottleneck. This is because most apps are not designed to queue requests if they don't fit in the available memory - they just allocate until they fail.
So, unless your apps are designed to queue requests in order to fit them into memory, this principle is probably not the explanation for the high memory usage.