Archive for November, 2010

Hiring principal level QA engineer: follow these principles

November 28, 2010 Leave a comment

Sometimes budget meetings simply work as intended.

Today, you received approval to build a new QA team with a primary mission to lead load and performance testing activities.

As counterintuitive as it might be, make the first step a very different one – take one step backward to make three steps forward.

Step one:  Clearly state what load and performance is and isn’t;  it is load and performance engineering, and not simply testing.

Step two:   Try to avoid hiring a manager and then ask him / her to build the team.   When superb senior talent becomes the foundation of a new team, these new hires set the bar for skills and performance.   Once on board, these senior hires should be given the opportunity to interview their future manager.   They will help you select the right manager.   Senior talent will not work for the wrong leader.

Step three:  instead – define a profile a principal level QA engineer and focus on recruiting, attracting, and hiring these rare individuals first.  The profile is simple and uses three guiding principles:

1.       First and foremost:   understand what is being engineered, how the architecture may have evolved / changed during the last N number of builds, how the functionality will be exercised by customers (large and small), different operating environments, and how all these factors influence a specific test approach

2.       Based on Item 1:  propose, structure, and lead the execution of complex tests without significant oversight

3.       Using a broad variety of tools and techniques:   execute tests, collected data, synthesize results, and provide meaningful and actionable feedback to Engineering team.  Meaningful and actionable feedback is not a report from a testing tool.  It’s an engineering assessment which will provide the Engineering team with the necessary data driven evidence to make software changes.  For example:

– Build 20 vs Build 19:  report A execution increased by from 2 seconds to 4 seconds;  query contributing to the increase is …;  root cause – new index is not being used

– Build 20 soak test;  after 7 continuous days of testing, a memory leak was detected in a new messaging agent;   details follows (message arrival activity, message types, detailed instrumentation data about which objects exhibit memory leak symptoms).

Principal level QA engineer is a true partner of software engineering team.   This person is an engineer first, not just an operator of specialized testing tools.