A/B Testing works fantastically for small, iterative, changes. It is not the correct tool for assessing sweeping design changes, however. Some sort of a focus group or large survey probably makes more sense for tracking something this big (which may cause substantial lateral shifts in user behaviour).