Not a joke / easter-egg. RELU6 is an activation function commonly used in deep convolutional neural networks. It comes up fairly often in mobile machine learning cases because it's used in Google's optimized MobileNet architecture and would cause errors when trying to convert to run on device.

The original paper detailing the function is here (PDF warning):

http://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf

"Our ReLU units differ from those of [8] in two respects. First, we cap the units at 6, so our ReLU activation function is:

y = min(max(x, 0), 6).

In our tests, this encourages the model to learn sparse features earlier. In the formulation of [8], this is equivalent to imagining that each ReLU unit consists of only 6 replicated bias-shifted Bernoulli units, rather than an infinite amount. We will refer to ReLU units capped at n as ReLU-n units."

It's in TensorFlow: https://www.tensorflow.org/api_docs/python/tf/nn/relu6

And even points to a paper: http://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf

I'm not sure why they didn't make it a variable maximum though, or what the advantage is over multiplying a sigmoid by 6.

There's an explanation here:

https://stackoverflow.com/questions/47220595/why-the-6-in-re...

It's an arbitrary value, but 6 fits best into 8 bits.