I am by no means an advocate of Mill's liberalism, but according to that philosophy espoused everywhere else but the case of a business refusing access to black customers seems to me inconsistent.