Right I understand the reason. But as my question says, I don't understand how a rocket inserts it into that orbit.

This video shows the steps that take it from a folded-up thing stuffed into a rocket to a deployed telescope at L2: https://www.youtube.com/watch?v=bTxLAGchWnA The rocket is out of the picture fairly early on in the timeline.

Whoa. A deployment sequence to put Curiosity's one to shame. Too many moving parts, over the course of a month. I hope all goes well.

It's easier to think of it not as "stopped", but in an orbit around the earth that takes exactly 1 year to complete. So it does have some tangential velocity.