Did you remember to put a vzeroupper instruction at the end of your AVX functions? If not then you pay a performance penalty when transitioning from AVX to SSE code, and the rest of your program might be inadvertently touching SSE code hidden inside the Go runtime.