Also, segregation spread from the South after the Civil War so there was a significant gap from the end of NY slavery to the start of Radical segregation which means it was not a continuous period.
Remember the Civil War ended in 1865 slavery absolutely ended in NY in 1827, but was almost non existent at that time long having been phased out.
However, at the time women where not allowed to vote at all. Thus it was not pure racial discrimination.
Just for perspective North Carolina and New York did allow freed blacks to vote in 1790. And in 1860 two states still had property ownership requirements to vote I am saying deep cultural views are a more complex topic than you think.
PS: As one historian put it, economics pushed poor white farmers to abolitionism before the civil war. However, economics pushed those same farmers to discrimination after the civil war.
Source: https://www.washingtonpost.com/news/made-by-history/wp/2017/...