Archive for March 14, 2010

Software testing: from elusive and insufficient to attainable and just right

March 14, 2010 4 comments

Even today, the state of software testing can be described as elusive and insufficient.

In short – insufficient software testing carries significant business risks.   Global enterprises invest millions of dollars developing & supporting mission critical software which should enable sales, marketing, supply chain, and customer service organizations to serve their customers better than  competition.

Another perspective at this multi million dollar investment: global enterprises literally fund and operate their own internal software company to gain a competitive advantage.

What if this internal software company just doesn’t do enough when it comes to software testing?  What is the true cost of insufficient software testing?  The answer is both obvious and sobering.

Because every functional area is enabled by a significant IT investment …

every functional area carries a higher cost due to insufficient software testing …

… leading to lower margins and the loss of a competitive advantage if critical IT projects are late.

Let’s ask (and later respond to) three questions.  These questions will help anyone determine the state of software testing in their own organization:

One:  do you design, build, and then test?  Or – think about testing after most of the development has already been completed?

Two:  how do you characterize individuals that perform software testing:  embedded & engaged? Or – disconnected?

Three:  who are they?    The best and the brightest? Or – second class citizens in the technology organization?

Let’s respond to question One.

Software testing – by definition – is an integral part of software delivery cycle.   Software testing must be integrated in all critical areas:

– Design:  should be questioned without any hesitation as early as possible by those who test.   How can this be tested?  Is there another design approach which might make it easier to create an automated test in the future?

– Process:  the benefits of writing automated tests during development cycle are well-known.  Set a realistic code coverage goal:  80% of all critical functionality must be covered by automated tests.   Monitor this goal during development.   Do not release software unless the goal is met.  Create and monitor measures of success to ensure individual accountability.

– Schedule:  schedule must reflect not only software testing but also all the different types of testing required.   Otherwise, schedules become unrealistic.    If the design specifies 10,000 transactions per hour, appropriate tests must be planned and scheduled well in advance to determine if this scalability level can be reached.   That’s why those who test should engage early as equal partners.

Responding to question Two is now much easier.  If those who test engage early as equal partners, they will be embedded and engaged.

I think it’s best to leave question Three unanswered so the reader can answer it themselves.

Instead I will briefly discuss changes that are essential to create the organizational culture where software testing is an equal partner.

– Recruit, hire and retain the best and the brightest for software testing roles

– Create a compensation model that rewards software testing excellence

– Define and practice software testing as a true engineering discipline, not just an exercise in writing scripts

– Make software testing an integral element of delivering great products