It can be difficult to evaluate web ops candidates, for a couple of different reasons. One is that the breadth of knowledge needed for the field can be pretty wide, so spending too much time on any particular technical area can be a waste of time. Another reason is that it can be difficult to gauge how collaborative someone’s demeanor is in an interview. Collaboration is a requirement at Etsy. 🙂
So in addition to the standard technical questions, I like to ask high-level questions where the answers can zoom in and out of a larger picture within the operations context.
- Diagram the current architecture you’re responsible for, and point out where it’s not scalable or fault-tolerant.
- What are some examples of how you might scale a read-heavy application? Why?
- What are some examples of how you might scale a write-heavy application? Why?
- Tell me how code gets deployed in your current gig, from developer’s brain to production.
- Tell the story of the best-run outage you’ve been a part of, in as much detail as you can. What made it “good”?
- Tell the story of the worst-run outage you’ve been a part of, in as much detail as you can. What made it “bad”?
- What is the purpose of a post-mortem meeting?
- How do you handle (and feel about) making changes (code/schema/network/etc.) in your current environment?
These are purposefully open-ended questions meant to dig into what’s important to you as someone responsible for the performance and availability of a growing website. This is just a snippet of what we normally ask, in addition to my (and Jesse‘s) favorite interview question.
So: maybe you should take a look at the type of ops engineers we’re looking for, and apply? 🙂
Good stuff! I’ve bookmarked this for further use since I suck at interviewing.
Open-ended questions are great, but the “favourite interview question” is part of a smartass demeanor that makes all alarm bells ring in my head about the company. Asking this sort of question really means: oh, look just how GOOGLY we are! Don’t you just WISH you worked with us? We’re so smart and we know all the layers and stuff!
Usually, when such question is asked I start getting into details of the keyboard and video drivers and the internals of web browsers. Smart recruiters get the message, drop the subject and start asking normal questions and getting normal answers. 🙂
Never ever asking nor answering that question really helps me to be a successful and collaborative web ops engineer.
Claude: it sounds like you’ve had some bummer interviewing experiences. Sorry about that, it’s happened to me before, too. 🙂
That question, which is basically: “describe what happens in order to get a web page delivered to you, from the browser to the server, and back”, isn’t some sort of abstract question designed to pump the ego of the interviewer, like you suggested. It’s meant to get at things that are important for the job. Preferably, in front of a whiteboard. Not a question asked by a phone-screening recruiter.
I want to know that a candidate knows what role DNS plays in a request. How the basics of the Internet works. What packet loss can do (or not) to certain requests. I want to know that they have awareness of load balancers and webservers and what they can do, what they can’t, and how they can break.
Do they include a CDN in their answer? That will tell me something about their experience. Do they focus on what PHP might do to assemble the page, or do they talk more about round-robin or least connections? That will tell me something about their experience. Do they first give a high-level answer, and then drill into more detail?
And most importantly: can they communicate these topics, ideas, and mechanics effectively and comfortably?
Those are the reasons I like the question, and there’s enough material in the question that can go in any number of directions. The question is meant as an obvious starting vehicle to explain experience and knowledge.
I’m not sure I would ever work any place who felt the need use interview questions as a means to express how awesome the company is, or really anything other than gauge whether the candidate was right for the position. I don’t like brain-teaser questions involving pirates and gold pieces or abstract ones like “how do you design a manhole?” I only like questions that pertain to the job, and that one above, does.
Are you in the US? Maybe you should apply and you can tell us about your experience. Although I’m not sure that refusing to answer questions in an interview would impress upon me your collaborative abilities. 🙂
Back in the days when I interviewed regularly we’d start with a phone interview consisting of about 25 straightforward technical questions with unambiguous answers. I can’t over-emphasize how important that was. It saved us hours of face-to-face interviews with unsuitable candidates. But they have to be questions that can be answered in a couple of words, with few alternative answers: then you can maintain consistency even if you share the workload of making the calls out across the team.
What I like about the “tell me how it works at your current job” questions is that the answers will reveal a lot about how much the interviewee thought about the system they currently work on. I can’t believe I never thought of just having a standard series of well thought-out questions like that; we’d spend some time picking up on details in the CV and asking more ad-hoc questions.
We’d start the questions with “what’s your favourite X? And why?”, where X was some tool that no Web Ops person could avoid using. Then we’d go into some scenario questions: “You have system consisting of blah and blah. You are told that ‘typically vague problem description’ is happening. What do you do?”
Ah, interviewing and recruitment, surely the single most important job of a manager. The one legacy that’ll last long after you’ve left the company. Where your mistakes will be around longer than your successes and it’s so much easier to reduce the quality of a team than to increase it.
The first time I was asked to interview someone, I was given about half an hour to prepare and we didn’t even discuss what we were going to ask first. I knew I was starting to get somewhere when the recruitment agencies started complaining that we were being “picky” or “fussy” and coaching candidates on likely interview questions.
And yet still I used to worry that I was recruiting in my own image: people who thought like me. I never quite knew how to fix that problem.
Claude, the “what happens in your browser” question is an excellent devops interview question and not part of a smartass culture. IMO, devops folk need to be strong technical generalists with a systems orientation. The beautiful thing about asking this question is that is so easily exposes these qualities. Conceptually, the answer is very simple. You can diagram it on the whiteboard and give a decent high-level answer in about two minutes. You can then spend hours diving into details.
(And yep, I’ve used it on every candidate I’ve interviewed in the past 5 years or so at $DAYJOB)
But let’s get back to the main point of this post; John’s trying to articulate interview questions that give him a sense of a candidate’s ability to collaborate (Well, he’s also trying to recruit some of us too 🙂 but that we’ll ignore that for the moment). In an interview, you could ask behavioral oriented questions like, “Tell me about a time where you successfully collaborated with engineering and operations staff to improve the performance/availability of your site.” But really? Lame. Pointy Haired Lame actually. It seems that John is choosing these particular open-ended questions to serve as fertile ground for understanding a person’s collaborative style. To be fair, these questions are really more about domain expertise, systems thinking, and process, but I think they serve the goal quite well because he is eliciting opinions in parts of the question (judgements of “good/bad”, how do you “feel about it”). After all, if you don’t hold opinions on something, can you truly collaborate?
Coming in a little late on this, but very interesting post. I’ve been a hiring manager for a long time now and getting the process down is important.
I did a book review of Joel Spolsky (Joel on Software)’s hiring book, Smart and Gets Things Done – you might find it interesting. It’s not ops specific of course but it has a lot of great points.
Open ended questions are good, but in my opinion you definitely want to anchor a lot of them with specific historical events – “tell me about a time when you.” A lot of people talk a good game but track record is more important. Behavioral interviewing has its place.
I mix them up between behavioral and open ended too – questions from our last batch of interviews include:
If you were running a large multitier Web system, how would you arrange things to allow for “zero downtime” application code, data, and system upgrades? (Looking for insight and experience both technically (load balancing, db clustering) but also process wise)
Diagram your current systems architecture (or an older one/vary it/make one up if that’s a secret) and explain it to me. Feel free and use the whiteboard. (Looking for communication skills as well as hands on tech experience)
Tell me about a time you were leading the introduction of a new technology (or significant upgrade) into your environment. How did you introduce the change to users, administration peers, management, and others? How did that go, and what could have gone better? (The number of sysadmins that are twisted loners is sadly high)
You get a call and someone on the end of the phone says “The Web site is down!” Walk me through how you’d proceed from there. (I then guide them with answers to their investigation – “I check and see if I can hit the site”; “No, you can’t hit it either, you get a 500 error.” if the first words of the answer are “I log into the server” I pretty much write them off right there.)
You are working operations, and there are major issues with the applications – your app server tier is crashing frequently for hard to define reasons. You are in full firefighting mode trying to keep the site up and have been for 24 hours. Some developers show up and say “We have a new code release to go to production now! No, it’s not for this, it just adds new features to one of the apps. But hey, who knows.” How do you proceed?
Frankly if I think someone’s weak on the tech side I ask them the “what happens when you go to hit an URL from your browser” question too. Sure, if you’re a big ol’ smartypants you can answer it in chilling detail. But 90% of interviewing is weeding out the guys whose resume looks like they have 10 years of ops experience but don’t know what HTTP headers look like.
On questions indicating “how cool you are” – of course Google style whizbang questions are stupid, but an equal part of the interviewing process is selling the candidate on your organization and giving them a glimpse into how you do things – telling them and letting them ask questions is good, but if some of your questions are tailored to your environment that’s fine too. You want a two way match; many interviewing processes identify people that would be good for the job, but then they take another offer or whatever instead of you landing them – or you hire them and discover that they hate your work environment. (The guy who refused to wear pants comes to mind…) You’re trying to make a two way permanent match, it’s not all about finding out about them.
Pingback: How to hire an Agile Admin « the agile admin
I second Graham’s point regarding phone screens using a template. It does save a lot of time, especially if you are not using a recruiter to screen people and get resumes from job sites.
I consider a hands-on test a must for ops candidates. We bring up a virtual environment with a few machines configured as a web server, a database, and a client workstation, hand candidates a task list and ask them to complete as many tasks as they can in 1 hour. Tasks include installing software, fixing simulated problems, querying database, checking performance and specs of machines, etc. It is an overhead, but it`s worth it, and having virtual machine templates makes this easy – we can just tear everything down after a candidate is done. Obviously we only do this for candidates we are seriously considering based on previous rounds of screening.
I also like asking the “remove one of the 50 states now and justify your choice” question. This helps to see if they can remain detached in a stressful situation, and how readily they prioritize given a number of bad options.
Finally, I recommend craigslist as a cheap and effective source of candidates in NYC. Did not see your ad there.
Pingback: Recruiting ops engineers « jrwils
Pingback: Training Organizational Resilience in Escalating Situations
Pingback: Training Organizational Resilience in Escalating Situations | Revolusionline
碟醫院資料救æ´ï¼Œ13å¹´æˆåŠŸæ•‘æ´ç¶“é©—ï¼Œä½ é€éŽä¾†è³‡æ–™æœƒç¶“å°ˆæ¥å·¥ç¨‹å¸«åˆ†æžï¼Œå°‡ç”¨æœ€å¥½çš„方法所幫您把硬碟資料復原