> if a structure passes through a configuration where the 3 atoms are collinear, the dihedral angle will discontinuously change by 180 degrees.

Here's a stab in the dark: Instead of using the dihedral angle $\phi$, why not use:

  - $\cos(2 \phi)$,

  - $\sin(2 \phi)$,

  - $\cos^2(\phi)$

  - $\sin^2(\phi)$

  - the pair $(\cos(2\phi), \sin(2\phi))$,

  - the homogeneous coordinate $[R\cos(2\phi) : R\sin(2\phi)]$ where $R \neq 0$.

A change in $\phi$ by 180 degrees won't have any effect then.