A good visual introduction to neural networks can be found here: https://playground.tensorflow.org

A parameter is a "weight" in this case (the lines drawn from neuron to neuron). The neurons are effectively runtime values or "activations." Parameters (weights) are updated during training and then set as constant during "inference" (also called "prediction").

There's unfortunately a ton of jargon and different groups use different words almost exclusively.