There are in fact bad outcomes of continued memory growth - firstly there is a catastrophic performance degradation when processes start getting swapped to disk, then when swap space runs out Linux's OOM killer starts killing processes in an attempt to free up memory. Kind of surprised you didn't know this.
Except that didn't happen, according to the post.
The post says: "Indeed, the processes grew from their initial size of 100MB, slowly, all the way up to 1GB before we killed them."
The important part is that they manually killed the processes before they let it start swapping.
Again, they didn't actually have anything bad happen. Bad: processes couldn't get memory. Not bad: I was using 3 GB of memory.
> we noticed that the memory of the celery process was continuing to grow.
Doesn't look like there was any bad outcome related to this observation. Was any process not getting the memory it wanted?