I nuked my LinkedIn a couple of months ago. I had the same fear, but I decided fear is is not a good guide. Looking at it rationally, I never got a position through LinkedIn and I am very happy with the positions that I got so far.

If I ever end up in a world where it's only possible to get a job using LinkedIn, I can probably recreate my profile, along with contacts, and recommendations in a week or so.