Conversely someone coming from a startup wont understand working in a large team where progress is more methodical (slower). They will seek to circumvent processes they see as slow or useless. Perhaps they will try to 're-engineer' towards things they are more familiar with. In the process they may cause a net loss for the entire team.
In general I'd say its easier to go up than down. If you're used to the functions of a large organization it would be difficult to scale that back (network engineers that can't work without Cisco gear for instance). Someone coming from a startup is likely to be adaptive, and I'd rather have a generalist in a large organization, than a specialist in a small one.
Although I'm from an IT background not Dev so YMMV. The limit of my programming is single page perl/python scripts.
I'd think that it's a net win to have experience about the other side of the table, and that your skills are actually worth more by having been 'on the other side of the line' for a while.
If you were interviewing a candidate that had never worked for a startup wouldn't you take that into consideration?
People that apply to a start-up usually do so because it offers them something they can't get elsewhere. Freedom, responsibility and a smaller world. They've self-selected long before they applied because they're usually willing to take a fair sized pay-cut in order to be able to join. That speaks volumes about their motivation and willingness to adapt.
If a candidate had never worked for a start-up and didn't throw up any overt red flags if they're the most suitable guy or girl for the job I'd definitely give them a shot at it.