1. It encourages everyone to be aware that we're here to help
2. It encourages learning, the number of request for help decrease over time once the experience and familiarity ramp up
3. It exposes everyone to different types of work.
There are exceptions of course. Our P0 bucket has a dedicated set of people that handle that, and they are hand picked because those are house on fire situations that need the experience. Its also the one we put the juniors on when they know the ropes and are ready to take on the critical tasks so they can advance themselves (its good experience for career and personal growth I feel).
The other thing i like about this is that as a manager, I can actively encourage behavior by claiming a ticket, or helping someone with a ticket. I really want that culture effect to happen from the top down.
"Rock stars" are downright dangerous, as are people who prefer to make things up rather than admit ignorance.
A new SRE doesn't need to know everything (and can't). But he absolutely needs to be curious and willing to ask for help.
We look at essentially, (but not unequivocally) these things
1. Proven experience and desire to learn is a must. One of the best i have ever worked with came from a place where they worked pilot projects, and had to manage all their infrastructure themselves as the developers. No certs, no formal SRE experience (IE, thats not why he was employed previously). One of the best. He loved the work, and I could tell he learned so much doing this. It doesn't have to be this extreme, but having a proven interest in this line of work is top priority.
2. Is their depth better than their breadth? It is correct that you need a LOT of breadth, however I value the depth first. I'd rather someone have say, a medium about of breadth on the different technologies out there and a lot of depth on core subjects, like container management (this happens...everywhere nowadays) or cluster management. I don't need you know every single implementation of this in depth though. I need you to at least know one implementation of this in depth. I can build on that.
3. Because of the first 2, I need someone who is team oriented, as always.
> core subjects, like container management (this happens...everywhere nowadays) or cluster management
Curiously, these are subjects which most Google SREs won't know much about. One team deals with all that stuff as a service so the rest of us can get on with something else.
What would I pick out as our core skill sets? Ignoring technology-specific details that won't apply anywhere else: troubleshooting a system that you don't understand (reverse-engineering it as you go), and non-abstract large system design.
Broad experience, depth on a few topics. It's not impossible at all, it just takes time.
(edit: Note this may have changed since I left the company in 2009. It's been a while!)
There is a reason that this is an impossible to fill role.
I myself have deleted the data file of a production mysql server, because I have no idea what I am doing. I need to call teammates in 12am to learn how to take care of the mess I created.
However, having it be policy to ask first then trade or have someone else own it means essentially as a team, we can own that issue with you as you learn. It helps a lot come crunch time.
Mistakes will always happen though. If they are genuine and not from a lack of caring or trying with an earnest/logical thought process behind it, it doesn't matter in the grand scheme. I'd rather have my entire production server go down on an error like that than deal with a situation where the person was just being negligent to their duties and wasn't willing to be humble.
In short, you'll be fine :)
I wrote down some thoughts about this recently: http://zalberico.com/essay/2017/02/21/asking-questions.html
For some things, you should be able to trust others.
It's funny, because it boils down to "don't lie" (to yourself nor to others).
"Can I handle this? What if I can’t?" But then I started to work the problem in front of me, like I was trained to, and I remembered that I don’t need to know everything — there are other people I can call on, and they will answer. I may be on point, but I’m not alone
It might be because I'm currently training people in this realm, and this is one of their biggest fears, or maybe because it was my biggest fear, but its so true. We're a team. We're here to help. At least if your SRE org is any good. Never be afraid to ask for help, and never be afraid to admit you don't know something or it might be outside of your comfort zone.
I'll take willing to learn and readily able to admit knowledge deficits over someone who doesn't any day of the week. Great book they're working on, great article on this. So many gems, but this one stuck out for me, and its pretty relevant to me right now.