10 Staff Qualification and Skills
This chapter discusses the professional and social competencies expected of testers and points out factors critical to a successful test team.
10.1 Individual Skills
The “perfect” member of a test team has excellent knowledge and comprehensive experience in several domains: he possesses profound IT expertise; has knowledge about, and experience in, the application domain of the application or test object under test; brings with him a thorough professional qualification in software testing; and has high social competency.
Only very rarely do we find people who fulfill all these requirements equally well. Depending on training and professional career, most candidates are highly qualified IT experts (IT specialists) or they have comprehensive knowledge in the application domain under test (application or technical engineers).
A banker, for example, will fully understand the finance-related aspects of the VSR vehicle financing system. However, since he probably possesses little or no IT-specific qualifications, we may assume that his ability to comprehend system behavior or to notice technical flaws and system anomalies is rather limited. In contrast, an IT specialist will have only limited understanding of the banking system and its processes and may perhaps not be able to recognize banking-related anomalies.
Test managers must take the different educational backgrounds of their team members into account and assign to members of their team tasks and roles that best match their individual profile. Furthermore, knowledge gaps are to be closed through appropriate training and qualification measures.
The IT know-how comprises not only aspects of software development but also software testing and knowledge in system administration. In order to be able to work at test levels close to development (component testing, integration testing) or to be able to carry out certain nonfunctional tests (e.g., load or stress test), we need testers that have the required technical test and development skills. The same holds true when it comes to tasks such as structuring, formalizing, and consolidating test ideas provided by the domain experts. Other typical IT domains in the testing area are tasks concerned with test tools and the establishment and maintenance of the test environment.
IT expert as tester
In general, we can say that professional experience in coding, software design, system analysis, etc. will give testers a better feeling for potential sources of defects and frequent coding or logic errors. A tester with longstanding experience in these domains will be more likely to detect corresponding anomalies than someone without this background.
However, IT experts are unlikely to have sufficient knowledge of the application domain. Hence, if they are working as testers, they must be willing and prepared to familiarize themselves with the respective application domain and to acquire the necessary knowledge.
Test managers must keep an eye on IT experts in their team because they tend to pay too much one-sided attention to the optimization of test automation, enhancement of the test environment, and similar, technically challenging tasks. There is a certain danger that they lose sight of the application that is to be tested and that test resources are not deployed as they should be.
Danger: in love with technology
Application or domain experts know the technical and operational background and context (business processes, workflows, work products/results, work instructions, etc.) of the system under test. They often have in-depth knowledge of nonfunctional requirements such as system usability or performance because they may have worked with a previous version of the system for some time. They have a good understanding of how the system performs its tasks, in which areas system failure or operational errors have severe consequences, and what normal system behavior should be. They are experienced PC users but not IT experts. They can assist in the specification and execution of technically relevant test cases and can therefore do a good and useful job in system or acceptance testing.
Application experts as testers
There is a certain danger of developing a blinkered attitude that may lead to the acceptance of complex or strange system workflows “because it’s always been that way” (in previous versions of the system). A domain expert can rarely judge whether a workflow or feature could be implemented by an IT expert in a simpler or more elegant way. Sometimes people just lack the courage to ask a seemingly stupid question or to show a weakness in front of IT experts.
Danger: blinkered attitude
Some companies consciously delegate user-oriented tests to “laymen”—for instance, by providing temporary part-time jobs for a group of students of different disciplines to execute the system tests. As a rule, these people have neither test nor user application-specific know-how.
Laymen as testers
This only works if the test specification is up-to-date and very detailed and if testing is accompanied and supervised in detail by an experienced test manager. In all other cases, this approach will yield suboptimal results. Especially when it comes to detecting failures, experience has shown that laymen do not have the necessary perception.
One of the few exceptions is the “usability test”. In this case it makes sense and sometimes it may even be imperative to work with “laymen”. However, tests are performed under guidance and follow a predefined plan, such as, for example, a usability test specification.
No matter which group a member of the test team belongs to, a sound education or training in the area of software testing makes sense and is in most cases necessary. All members of the test team should have at least obtained the “Certified Tester – Foundation Level” certificate. Some members require additional, role-related in-depth knowledge provided by the “Advanced Level”—for example, the “Certified Tester – Advanced Level – Test Manager”. This also applies to testers with long-standing experience, since this is the only way to guarantee that test terminology, for instance, is interpreted in the same way and that people develop a uniform understanding and comprehension of the methodology.
Software test training
In order to be successful and in addition to his IT-, application-, and test-specific skills, the tester is required to have social competence. The following are some important and helpful character traits for the tester:
Social competence
The ability to familiarize himself quickly with complex domains and applications.
The ability to detect defects. This presupposes a certain basic skepticism and willingness to challenge apparent facts, paired with creativity, curiosity, and analytical thinking (“professional pessimism”).
The ability to cope with and voice criticism adequately (political and diplomatic aptitude).
The ability to distinguish essentials from nonessentials and the courage to leave out what is less important (“courage to leave some gaps”).
Discipline, exactitude, patience, perseverance, frustration tolerance, determination.
The ability to work in a team and ability to communicate.
Although the ability to work in a team and to communicate are listed last, both qualities are highly important. Because testing is teamwork, only those who are able to work in a team and communicate with colleagues and customers alike will have lasting success in their work as testers. The following sections will look at the different aspects of the test team work in more detail.
10.2 Functional Team Roles
The different tasks in the test process are reflected in the different roles within the test team:
Test manager: The test manager leads the test team. He is responsible for the creation of the test schedule and its technical and on-time implementation. He reports test status and test results to the project manager, product manager, or development leader. The test manager is experienced in test planning, test control, and test process improvement. Of course, he has knowledge and practical experience in general methods of software testing. It is also desirable that he should have knowledge and experience in quality management, project management, and human resource management. And last but not least, he should have knowledge of applicable standards and of corporate- and product-related software development and test processes.
Functional team roles and qualification profiles
Test designer: The test designer is responsible for the creation and maintenance of test specifications. His role involves identifying the appropriate test methods and the definition of a suitable test environment. He supports the test manager in the creation of the test plan and test schedule. He requires know-how in the areas of software testing, test specification techniques, and general software engineering. He must be able to familiarize himself quickly with complex application domains, requirements documents, functional specifications, and system prototypes. Furthermore, he must be able to comprehend the function and expected behavior of the system under test. Using this information, he derives appropriate test cases and documents them in such a way that they are fully traceable. One of his most important character traits is that he is able to distinguish important from less important information and set priorities. Some practical experience as a tester is necessary.
Test automator: The test automator is the test team’s programmer. He has comprehensive programming experience, general software engineering know-how, and a very good knowledge of the test tools and script languages applied in the project. Basic knowledge of testing techniques and practical experience as a tester are of great advantage. With sufficient experience in test automation, the test automator can specialize further to become a developer of test frameworks and test tools.
Test administrator: The test administrator is responsible for the installation, operation, and maintenance of the test environment. Among other responsibilities, this involves installing and setting up the system software (operating systems, database systems, application server), installing and configuring the test object, installing and setting up the test tools, and creating, managing, and restoring system configurations (images), e.g., by means of imaging or virtualization software.1 The test administrator has necessary system administrator know-how (see also section 5.2.11). Since he is a troubleshooter and firefighter who can solve complex installation or configuration problems even under stress, he is an indispensable member of the test team.
Tester:2 The tester is responsible for the execution of the tests and the documentation of the test results. He executes the test cases according to the test schedule and test specification. If he notices a deviation from the expected system behavior, he writes an incident report. The necessary qualifications for this job are test fundamentals, IT basics, ability to operate applied test tools, and a basic understanding of the test objects. The tester’s work style is characterized by concentration, accuracy, and perseverance. In addition, a good tester must be able to go beyond the limits of a test specification and apply his own ideas and creativity in a systematic manner. Depending on the test specification’s degree of detail and the maturity of the software under test, it is almost always necessary during testing to fill gaps in specifications and to add additional test cases in an ad hoc fashion to isolate presumed or identified failures. A good tester also distinguishes himself in writing brief, succinct incident reports containing correct and traceable information.
Experts (load test expert, database expert, network expert, etc.): Their duty is to support the “core” roles mentioned above in technically sophisticated matters or in problem solving.
Ideally, each of these roles is exercised by a specially trained member of the team. If in smaller teams several roles must be combined into one, the following combinations are best suited: test manager/test designer, test designer/tester, test automator/test administrator.
Role combinations
10.3 Social Team Roles
Besides their professional role, be it consciously or unconsciously, willingly or unwillingly, each team member assumes a social role in the team. The role one plays in the team depends on the individual’s personality, his experience in his role, and how well he can fill it. [Belbin 93]3 distinguishes nine typical team roles (see table 10-1).
Table 10–1 Personality types4 according to [Belbin 93]
Type |
Descriptors |
Strengths |
(Allowed) Weaknesses |
Monitor Evaluator |
Sober-minded, strategic, and discerning |
Good power of judgment, discrete, |
Lacks drive and the ability to inspire others |
Shaper |
Dynamic, open-minded, edgy, result oriented |
Lots of drive, fights idleness and inefficiency, exerts pressure |
Prone to provocations and irritations, prone to hurt others |
Planter |
Individualistic, unorthodox |
Brilliant, creative, lots of intellectual power |
Often absent-minded, inclined to ignore practical details and instructions |
Completer |
Meticulous, straight, conscientious, anxious |
Ability to see things through to completion, perfectionism |
Inclined to worry unduly, reluctant to delegate |
Team worker |
Cooperative, mild, sensitive |
Ability to cope with different situations and people, promotes team spirit, calms the waters |
Indecisive under pressure, can be easily influenced |
Implementer |
Conservative, dutiful, calculable |
Hard-working, turns ideas into practical actions, disciplined |
Somewhat inflexible, rejects unproved ideas |
Co-ordinator (chairperson) |
Self-confident, mature |
Quick at recognizing individual talents in team members, knows how to use their strengths unreservedly, good at clarifying goals |
Not necessarily a high-flyer, has limited creativity |
Resource Investigator |
Extrovert, enthusiastic, communicative |
Likes to develop internal and external contacts, picks up new ideas, reacts to challenges |
Overly optimistic, inclined to lose interest after initial enthusiasm |
Specialist |
Single-minded, self-starting, dedicated |
Provides knowledge and skills in rare supply |
Contributes only a narrow front, dwells on technicalities, overlooks the “big picture” |
Of course, the specific features or characters of real people are much more complex and multilayered, but knowing about these personality profiles may help when it comes to employing new members that need to “fit into the team”. It also helps us understand the success or failure of an individual team member or even a complete team and may help improve team performance.
If new team members are to be selected or employed, care should be taken that already available knowledge and personality profiles are sensibly complemented by the new team member. Belbin’s investigations on group psychology show that regarding the different personality profiles, lopsided teams work5 less successfully than properly mixed teams.6 Teams, for example, that are entirely made up of “Implementers” will only perform well in the completion of routine tasks. However, when it comes to solving complex issues, such teams will lack the ingenuity and skills necessary for creative problem resolution.
Selection of team members
Conversely, if we have too many “Planters” in the team, it may be extraordinarily creative but will lack the ability to implement and utilize its ideas with the necessary perseverance and endurance. Teams composed entirely of people with exceptionally high intelligence (Belbin calls them “Apollo teams”) do not necessarily turn out to be successful. Such teams are difficult to control; they are prone to destructive debates and have difficulty making decisions. Despite a whole bunch of highly intelligent people, the team flops.
It is therefore important to find a suitable combination of different characters or personalities; it is equally important that the distribution of tasks in the team is appropriate to the test constellation. Especially when it comes to team forming for software testing, it is difficult to get professionally qualified staff. In that case, selection based on somebody’s “personality profile” is of only secondary importance. However, it can help in the correct distribution or, in case of problems, redistribution of roles or tasks in the team.
Getting the task distribution right
Examples: task distribution
It is certainly of advantage if “Planters” or “Monitor Evaluators” work as test designers. Yet testing in complex system environments or testing of “immature” systems, too, asks for “Planters” with analytical skills. They should at least be available as coaches because as soon as technical problems arise, their know-how and creativity is required.
Completers, Implementers, and Teamworkers will feel comfortable working as testers, especially in regression testing. A “Planter” will find this a rather boring affair. Consequently, he may not observe the test specification to the letter and will try to find his own test variants. This, however, is not required in regression testing.
Despite all these considerations, we must not forget that the personality types just listed are “constructs” only and that in reality we hardly ever encounter them in pure form. One and the same person can quite often slip into several roles, for instance, if the preferred role in the team is already occupied. The role concept and the idea of personality profiles help us in our understanding and analysis of problems in the team or of the reasons for low achievement and weaknesses of individual team members as a whole.
Stop thinking in stereotypes.
Care must be taken not to carelessly pigeonhole people. Other factors, too, ought to be taken into account, such as personal antipathies, career thinking, or factors pertaining to the team as a whole, like company climate or current project situation.
10.4 The Communication Factor
Testing does not only involve finding and documenting as many defects as possible. It is equally important to communicate identified problems or defects in the right way and to the right people. If there is interest in quick defect resolution, the way a defect is communicated is often more important than the content of the report itself.
Communicate problems accurately
Test-Team-External Communication
Generally speaking, members of a test team and especially the test manager have to communicate with three main team-external groups:
Communication between testers and developers:
A tester detects a failure in the software or in a colleague’s work result. He writes an incident report and thus lays open the defect in more or less forthright terms. The ways in which the incident report is written must always be factual and problem oriented. Personal or unobjective criticism or even finger-pointing must strictly be avoided.
Tester → developer
A diplomatic communication style is an indispensable precondition for the acceptance of incident reports (hence, testing as a whole) as a positive contribution to the project effort and is the only way to contribute to the improvement of the product and its quality.
The same applies for communication in the opposite direction. Nobody can “blame” the tester for the fact that failures occur, and he is entitled to receive a well-conceived answer to each of his reports. It really makes good sense if developers inform testers about which system parts require particular attention—for instance, because of their high degree of complexity or because they are newly conceived. Conversely, it helps if testers pass test cases on to developers in support of early testing within development.
Developer → tester
Example: “Well” and “badly” formulated Incident reports
The following examples illustrate a “well” and a “badly” formulated incident report.
Bad Style:
“... Tried to save a purchase contract but application still crashes. This has been known for weeks; see reports 264 and 253. Can’t somebody come up with a decently coded program?...”
Good Style:
“Crash after attempt to save purchase contract:
Description: ContractBase crashes when trying to create new purchase contract.
Impact: Contract data is lost; system needs to be rebooted.
Reproduction:
Call up ContractBase
Select customer
Select new purchase contract (compare attached screen shot)
“Save contract”
→ crash
...”
Communication between testers and project management: The test manager regularly reports test progress and observed software quality to product or project management. The test (status) report needs to be open and direct. The same holds as before: solution-oriented, constructive contributions and formulations will facilitate things. Statements concerning individual team members and personal finger-pointing are not acceptable.
Test manager → project manager
Conversely, product/project management informs the test manager about changes in the project plan and delivery dates; new, changed, or deleted features; system environment changes; staffing changes in the development team; and new software suppliers.
Project manager → test manager
Communication between testers and users: Testers, and in particular test designers, should try to stay in close contact with users, developers, and other stakeholders involved in the system under test. Requirements specifications are hardly ever detailed, complete, or up-to-date enough to do without additional talks with the people who originally wrote them. Here, additional background information may be gained about the system’s application domain or information that will help set testing priorities. In order to prioritize, remove, or contain failures it may be necessary or helpful to explain test results to system users (e.g., pilot customers).
Tester → user
Test-Team-Internal Communication
Besides external communication, the test manager must also pay attention to team-internal communication. Scheduling regular team meetings is an important measure in support of team-internal communication. In order to be efficient, the meeting must have a fixed agenda and must be moderated. In practice, the following agenda has proved useful:
Test team meeting: agenda
General project situation
Current status of test progress versus plan
Test object quality, defect rate, defect correction percentage (bug fix rate)
Release planning and upcoming release test cycles
Current staff planning
Necessary changes to the test schedule
Quality of the test process and improvement potential
Status tracking of other tasks outside the test schedules (e.g., expansion of the test environment)
Newly identified tasks documented and assigned to a member of the team
10.5 The Motivation Factor
Effective and productive testing not only requires the right technical and social skills, it also requires the full commitment and motivation of the testers involved. No matter how good the process, and no matter how well people are trained or how experienced they are, without the right motivation the end result will be bad or at best mediocre.
How can motivation be achieved and how can it be sustained at a high level? What are the factors that can dampen or even kill motivation altogether? “The motivation checklist” [URL: Templates] contrasts motivating versus demotivating factors. Using this list, it may be helpful to once in a while check the motivational situation in the team.
Motivation checklist
Of course, not every circumstance that may have a demotivating influence on the tester or the test team can be influenced or even eliminated by the test manager; however, there are many possibilities or opportunities where he can actively intervene.
Some of the measures that a test manager can take are listed below:
Plan realistically. Communicate clear objectives and provide for a clear task distribution. Track the plans and respond to deviations.
Ensure management support. Promote testing and show the benefits of testing to everybody. Present test results regularly and make sure they’re well prepared. Present the costs of testing in an unvarnished and honest way and compare them with the benefits of testing (see [Spillner 07]).
Provide for a professional, adequate working environment (facilitating communication, test laboratory). Provide for interesting tasks that will relax the inevitable test routine. Allow and encourage specialization within the team. Point out opportunities for further personal development and careers.
Provide feedback to the test team about its contribution to the project’s success or product quality (by way of feedback from management, development, support, and the market).
“Propagate” the accomplishments of the test team within the organization. Present the successes of the test and development teams as a collective success that will be collectively celebrated.
10.6 Summary
A member of the test team needs to possess IT knowledge, knowledge of the application domain of the system or test object under test, and knowledge and experience in the area of software testing.
In addition, in order to be successful, a tester needs to have social competence.
The different tasks in the test process are reflected in the different roles within a test team: test manager, test designer, test automator, test (laboratory) administrator, and tester.
Besides their professional role, each team member also plays a social role in the team. Not only professional qualification but also the suitable combination of different personalities and a suitable task distribution within the team are vital to team success.
Detected problems or failures must be communicated to test-team-external addressees (developers, project management, users). The style of communication must be objective, solution oriented, and focused on improving quality. Personal or unobjective criticism or finger-pointing must be discouraged.
Regular test-team-internal meetings summoned and chaired by the test manager contribute to good communication.
The test manager is to provide for a motivating working environment and professional working conditions for his team. Part of this task is to adequately represent the role and contribution of the test team within the organization.