The practical answer is that the question is ill posed, and you need to clarify what is actually being asked. A reasonable way to answer the question as asked would be to model the coastline as a fractal, which would give you an estimated length of infinity. What is needed is not a better way of estimating the answer to the question as asked, because the answer to that question is not useful. Instead, what is needed is a better question (which would require knowledge of why the question is being asked).