Of course, if two or more people are involved and only one has a change of heart and stops actively participating (but doesn't take active steps to foil the plan in motion) the one who backed out is still (along with all the others) guilty of conspiracy. And that's true even before anyone gets to the point where they could be individually charged with an attempt, as long as one overt act toward the criminal goal has been done, the whole group is guilty of conspiracy.
And in the US, certain acts are often criminalized specifically when done with the intent to support some other target crime which is not negated by later backing out of the target crime (and in fact allows you to be arrested and convicted when you never got further than the preparatory act.)
Consider the following cases:
Arresting the guy who kidnapped someone.
Arresting the guy in the act of trying to kidnap someone.
Arresting the guy who set out to kidnap someone.
Arresting the guy who is preparing his home to hold a kidnapped victim and is identifying a target.
Arresting the guy who researched how to kidnap someone.
Arresting the guy who has expressed strong interest in kidnapping someone.
Which of these is wrong? I think we would generally agree the last two are, but which of the middle ones do we draw a line at?