Investing in startups is not like buying public stocks, where if you buy at the bottom of the market you're almost guaranteed a positive return. It's exactly the opposite, in fact. The returns depend almost entirely on which startups you pick, and hardly at all on market conditions when you invest.
It was not panic so much as it was an exercise of common sense by the VCs at the time. A lot of startups had been riding on fluff and, when the fluff went away, it was time to cut investment losses.
A crude, but key, indicator is that the liquidation preference in the relatively few fundings that did occur went from a former standard of 1x out to a new one of 3x to 5x. Few deals were done, and those that were done were on terms perceived by founders to be onerous.
In such a climate, it likely would have been deemed reckless conduct by the VC's managing partners not to cash in on the convertible note and to opt instead for an alternative outcome that was highly speculative at best. We can't really know without being privy to inside details, which we are not. In any case, that decision shouldn't be second-guessed by outsiders today.
Growth: http://i.zdnet.com/blogs/zappos-revenue.png
YC's 'How to Apply' recommends against saying you will win because your app will be well-designed and easy to use because presumably that is what everyone tries to achieve. Zappos eventually succeeded because they made the online shop equivalent: efficient back-end, excellent customer service, lots of products that people want to buy, low prices.