This is a pretty good paper, & they bring up many reasonable points, but I think it's important to distinguish deep metric learning from more traditionally formal ml methods for metric learning - there is plenty of progress being made in the context of scalable & provable metric learning algorithms that are robust to noise/corruption & missing data.

Recommend work & talks by Anna Gilbert for anyone interested. Entertaining & good at distilling technical content. Here is her most recent one, but there are other good ones on youtube. https://www.youtube.com/watch?v=Sb1ZhtsZjyM