Mammalian brains are so densely packed that we don't really know what their internal architecture actually looks like--that's why estimates of the number of neurons keeps increasing over time [1]. Most of our models are based on some autopsies, and rely on things like fMRI or electroencephalography (not often you get to spell that word!) to work out which portions of the brain are involved in certain tasks. We have no hope of being able to build a connectivity graph between neurons at our current state of technology.
The short answer is that the cerebral cortex is usually identified as having 6 layers. These can't be correlated with layers in a neural network, however. I do seem to recall a spitball number of 30-ish in relation to the visual cortex, but I've been unable to find any references as to what that number would actually represent, let alone if it's been superseded.
[1] The current estimate is around 100 billion. If you want to know the number involved in, say, the visual cortex, that appears to be on the order of a few billion.
That short answer is very misleading. Those 6 layers are histological layers of density and structure in the cerebral cortex (eg how the cross sections look a little different). You might have many (30+) layers of cells with high interconnectivity in each of those 6 layers. Current estimate is 100 billion for the number of neurons, but the estimate of inter-connectivity is on the order of 100 trillion synapses. The human brain complexity many orders of magnitude higher than anything we have ever "simulated" in silicon.
The brain's layers are both much more loose in their allowed interconnections but also more modular in their functionality. There are different visual areas (usually 5) at different locations of the neocortex that connect to each other (feedforward and back) in complex ways. As residents of the neocortex, each visual area will thus have 6 layers which are in turn dividable into sublayers (often 1 - 4). These layers are structural but are also partially organizational tools for the scientists.
There are several reason why this model just does not apply to the brain[1]:
1. The brain is not organized in layers this way. Instead there are interconnected circuits, feedback, brain regions, bidirectional connections, and such.
2.The brain's connections are feedforward only without "error feedback".
3. The brain doesn't learn from training data with correct answers (or a "ground truth").
4.The neurons in the brain spike and use different learning algorithms.
[1]https://www.quora.com/How-many-hidden-layers-are-in-the-neur...
What is had in common, and what was inspirational were (harsh) experiments on kittens, done in the 60s which provided hints as to what concepts were important in the primary visual cortex. These experiments lead to ideas of pooling and averaging as ways to calculate invariance, which made their way into the neurocognitron and then the convolutional net. The concepts of thresholding units and summation of inputs also played a part in the designs of Artificial Nets more generally. Most recently, new artificial NN designs have been loosely inspired by the importance of attention in biological networks.
Another similarity, but this is a general computational similarity, is the importance of hierarchical organization in easing learning difficulty. But unlike in the mostly feedforward networks we're used to in ANNs, brain's networks are also feedback, or with backward projections, across areas and up and down hiearchies and so on.
But there is more to vision than V1 and V4. The visual cortex has more arbitrary connections across layers and is not restricted to the bipartite structure common in MLPs and is more fine grained than the hidden layer summation across timesteps of RNNs.
Other things the visual cortex computes but not well captured in models: speed and directional selectivity, nontrivial color transformations, temporal frequency, global v local + foreground v background, optical flow, motion tracking, inhibitory neurons, higher level scene understanding, feedback connections, perceptual color constancy and contributions to memory formation.
Of course there are plenty of other things that the visual cortex is involved in, as far as we can understand it at least. I'm talking purely about the macrostructures that are involved in elementary vision functions like orientation.
I'm in the camp as well that conflating brains and ANNs is a dangerous and hype-prone line to walk, but I'm not sure you can categorically say that the layers in V1 and the layers in a CNN have nothing in common, with respect to the computational capabilities they model. Likewise with convolutions and columns.
However, I doubt that the layers in V1 and CNN have much in common for a couple reasons. First is because the layers in V1 will differ depending on who you're reading from and when they wrote it. The layer divisions are more structural than functional, so far as we know. You can't trace ever more abstract features as you move across V1 layers, not least because V2-V4 are also involved in this.
Next is that V1 sends projections to "higher" and also "lower" areas while connecting to other areas than just the visual. Within V1, connectivity from lower to higher layers are not obvious. Some inputs from the LGN might feed into layer 4 and others to differing layers depending on whether the computation is motion or color etc specific. Layer 5 might also get connections from more distant areas. This level of interconnectivity suggests that very different computations are going on even at the level of just V1. The connections are finer and more intricate than simple bipartite divisions. Basically, they're working out more than just features that are robust to rotations and translations (e.g. speed, direction, dual stream signal integration, oscillations...).
The prevalence and importance of feedback and recurrent connections is not yet well understood but are believed to be important in the sort of high level scene understanding that are still open problems.
And related to that, the notion of inhibition is turning out to be more and more important in biological networks though they aren't really key to any neural nets. Some of their repeating patterns or motifs, have a place in directional and size selectivity but more intriguingly, are figured to be key in the top down flow of information that allows for our (mammalian) complex scene understanding.
For example, we get some translation-invariance for free with the ability to move our eyes.
[1] http://ilp.mit.edu/event.jsp?id=6094 (I had the video URL bookmarked, but it's broken now)
There are feedback and recurrent connections in the brain but what (2) means to say that the specific mechanism is not biologically plausible. I know there are prominent Deep learning researchers who are trying to work around this but those explanations have never struck me as particularly parsimonious. It's better to start an explanation from experimental data on brains than from them most important aspect of neural nets.
Recurrent neural networks are incredibly hard to train with the exception of liquid state machines (also known as echo state networks or reservoir computing).
The human brain is also very efficient so it also makes a very good use of those neurons, and modern networks aren't. However, a human brain trains over a very very long time and also prunes itself constantly, while such efficiency techniques at scale for very large networks are being explored in deep learning only recently.