TESTING 371
game must adhere. The console manufacturer will provide a complete checklist
of each requirement the game must follow, and in some instances will provide
tools to assist in checking the title for compliance with the appropriate require-
ments. Noncompliance with these requirements puts your game at risk of not
being approved by the console publisher. If this happens, and the error is not
fixed, your game will not be approved for release. Therefore, testing for compli-
ance to the requirements is extremely important. Please refer to Chapter 23,
“Code Releasing,” for more information on the console submission process.
Designate a single person on the QA team to be responsible for knowing
the technical requirements and how to test them in the game. If this process is
centralized under one person, there are less likely to be mistakes when the game
is checked against the requirements. It is very frustrating when a game fails sub-
mission because incorrect terminology was used, or a required error message is
not present. This tester should have extensive knowledge and an understanding
of how to interpret the technical requirements, and be the main point of contact
with console manufacturer if there are any questions about the requirements.
Finally, this person should be familiar with how to use the proprietary develop-
ment tools provided by the console manufacturer that automates the checks for
many of the requirements. Any time a non-compliance bug is entered into the
database, it should be rated a high priority and addressed as quickly as possible.
While it may not be a crash bug, a non-compliance bug is as serious as a crash
bug, as it is possible for the game to fail submission if the bug is found during
the approval process.
22.6 EXTERNAL TESTING
At some point in the project there may be a need to outsource the testing to an
external vendor. This usually occurs when the developer has a very small testing
team and wants a second pair of eyes to thoroughly test all areas of the game.
External testing also occurs when the localized versions of the game need to
be tested by native speakers of each language. Finally, money might be better
spent on external testing if there is a specific area to test that requires specialized
resources. For example, checking PC compatibility on a variety of computers,
video cards, and sound cards, might be best handled by an external vendor who
has a compatibility lab already set up for testing. It may be worthwhile to hire a
testing vendor who has extensive experience checking technical requirements in
order to ensure your game has the best chance of passing the console submission
process in a timely fashion.
If you are planning to use external testing vendors, be sure to do your
research on the vendors and secure recommendations from other satisfied
372 THE GAME PRODUCTION HANDBOOK, 2/E
(or unsatisfied clients) when you are able. Things to ask the vendor about
include:
■
What is the best way to get them a build of the game—will they set up
an FTP site for you to use, or do you need to mail them discs?
■
How are the testing rates determined—do they charge by the hour, or
by the day?
■
How far in advance do they need to be notified if you want to cancel
testing—do they require 24 hours notice?
■
Will they write a test plan—will they create their own version of the test
plan, or will you need to provide one for them? If they write the test plan,
find out how much this costs. Also, keep in mind they will need design docu-
mentation and playable builds of the game in advance so that the test plan is
ready when they begin testing the game in earnest.
■
What additional costs are adding to the testing rate—do they charge
a project management fee, or tack on an additional percentage of the total
testing cost for overhead? You don’t want to be surprised when you get the
bill.
■
Can they provide an estimate of how long it will take to test the
game—will they do a free gameplay evaluation in order to determine how
much testing time they need to in order to fulfill your request?
■
Who will be the main point of contact—will they provide a project man-
ager that you can contact on a daily basis and who will send daily progress
reports?
■
What bug tracking software do they use—do you have to use their soft-
ware, or can they accommodate other types of bug-tracking software?
Once a vendor is selected, you will need to do your part to get the most out
of the testing as well. For example:
■
Check builds before sending them to the vendor to ensure they in-
stall and load correctly. If the vendor has to spend a few hours trouble-
shooting a broken build, the costs can add up quickly.
■
Provide clear direction about what needs to be tested. Are they only
focusing on the console technical requirements, or do they need to test other
areas of the game as well. Are they only supposed to check 16-player mul-
tiplayer functionality, or are they also supposed to check the single-player
campaign. Are they only focusing on regressing bug fixes from the previous
build? If so, be sure the bug database clearly indicates which bugs are ready
for regression.
■
Establish a schedule for sending new builds. Are they supposed to
test a build from start to finish and then wait until you are ready to do
TESTING 373
another round of testing? Are they supposed to continually test for the next
two months—if so, how often do they need a new build (every day, every
week)?
■
Answer questions as quickly as possible. If the vendor has a question,
get them the necessary information as soon as possible, as testing may be put
on hold until the question is answered. The longer the wait, the more testing
time is lost, and the more money is wasted.
22.7 CHAPTER SUMMARY
Testing is a time-consuming and stressful aspect of game development. As you
are trying to get the game out the door, the tester might find a crash bug that
was not uncovered earlier in development. If this happens, tensions run high as
the developers scramble to prepare another code release candidate as quickly as
possible. This scenario is bound to happen on some game development teams,
but if the producer, leads, and team are constantly keeping the QA needs in
mind during production, some of these instances can be avoided. This chapter
discussed general information on how to work effectively with the QA depart-
ment from pre-production to code release. Topics included the testing schedule,
the testing cycle, and closing bugs.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.