You can beat the pants off this, even though it beats the standard proto library. Most of the time here is being spent calling out-of-line functions through Go's stack calling convention because the Go compiler hates inlining. For example, DecodeTagAndWireType is CALLing DecodeVarint, even though that's really where you want it to have been inlined.

If your use case is really simple, and you want the fastest possible proto decoding, you can hand-roll it pretty easy.

  message point { fixed32 x = 1; fixed32 y = 2; }

Then your code is structured such as

  var x uint32
  var y uint32
  for len(buf) > 0 {
    switch buf[0] {
    case (1<<3) & 5: // Field 1, type fixed32
      x=binary.LittleEndian.Uint32(buf[1:])
      buf = buf[5:]
    case (2<<3) & 5: // Field 2, type fixed32
      y=binary.LittleEndian.Uint32(buf[1:])
      buf = buf[5:]
    default:
      // Freak out somehow
  }

Improve the exception hardness of this code to the extent that it suits your use case.