Consider BUGS as it's a very prototypical probabilistic programming example. If your goal is to specify that you have a model where you observe the "wetness of the grass outside" and infer whether it "rained previously" then you can use BUGS to express it easily.

    model {
      rained ~ dbinom(0.1) /* prior probability */
      for (i in 1:N) {
        grass[i] ~ rained
      }
    }

(That was a very sketchy example, don't read into it too closely)

The heart of this is that quantitative methods are getting so well understood that we can express the algebra of models as a formal language. When we do this we get many of the benefits that programming has done for formal expressions of flow-charts.