In practical terms, employees do have input at hearings in which an employer will challenge the results of an audit because it is the employees who are often the key witnesses in such hearings (this usually works to the severe disadvantage of employers though, as a parade of employees is often called in and asked, "did you have to report in for work at a specific time?", "is it true that your duties consisted of making telemarketing cold calls while strapped to a desk?," "are you telling me that making such calls non-stop from 8 to 5 every day signifies that you were 'in your own business' just because your boss made you get a business license?" - this often goes on and on to disastrous effect for the employer and, though I have framed it a bit facetiously here, this is about the spirit in which it occurs).
Therefore, if you try to claim you are a contractor, this will be given only the slightest weight, if any at all - in other words, in your words, they don't have to stick with what you say, and indeed normally won't (they will just apply their multi-part test and come to whatever determination they feel is justified).
On the other hand, if you claim you are an employee, they will give that a lot of weight and will ask you considerable detail about all the factual points that tend to confirm that you are an employee (in order to nail your employer with the taxes/penalties that result from the reclassification).
Everything is basically stacked against the person claiming to be a contractor. You can win on the issue but it is usually a tricky proposition. In most contested situations, you might win in two or three out of ten at most.