I work at a large tech company. I have to maintain a certification for human experimentation with my IRB. Usually, A/B testing falls outside of the experimentation framework as the actual feature in question cannot cause harm, is engaged with voluntarily, and users are already aware that the software changes arbitrarily (usually through marketing material as "continuous updates"). We usually only get into the human experimentation bit when we send out a survey or the like.

Is said a/b test supported or funded by the Department of Health and Human Services? If not, that website clearly indicates that the regulations do not apply.