If you set up an LLC elsewhere and want to qualify to do business in New York, you still have to publish notices (http://www.dos.ny.gov/corps/llcfaq.asp#pubreqreply).

If a company incorporates as an LLC in another state, New York loses out on the formation filing fees. Also, if the company also chooses to set up its office elsewhere, New York would lose out on additional tax revenue.