It means ritualistically imitating some advanced technology and thinking of it as magical way to get wealth and prosperity (based on primitive tribes building fake-airports and thinking that will bring the associated goods westerners coming to their land with airplanes had). And yes, it can involve all kinds of "rain dances or other backwards stuff" too.
"The name derives from the belief which began among Melanesians in the late 19th and early 20th century that various ritualistic acts such as the building of an airplane runway will result in the appearance of material wealth, particularly highly desirable Western goods (i.e., "cargo"), via Western airplanes."
They didn't imitate it "for the sake of it", they genuinely thought they would receive cargo by building runways.
So while it's not about rain dances and shakalaka sticks the previous commenter was on the money about the term cargo culting.
Explained the upvote because I don't want you to be confused.
It didn't rain? Must not have shaken the stick hard enough.