Key points

Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Key points

• This chapter contains descriptions of 24 standardized questionnaires designed to assess perceptions of usability or related constructs (e.g., satisfaction or usefulness).

• Those questionnaires fall into four broad categories: post-study, post-task, website, and other.

• Standardized post-study questionnaires include the QUIS, SUMI, PSSUQ, SUS, USE, and UMUX.

• Standardized post-task questionnaires include the ASQ, ER, SEQ, SMEQ, and UME.

• All of these post-study and post-task questionnaires are of potential value to usability practitioners due to psychometric qualification indicating significant reliability, validity, and sensitivity.

• Head-to-head comparisons of the methods indicate that the most sensitive post-study questionnaire is the SUS, followed by the PSSUQ; the most sensitive post-task questionnaire is the SMEQ, followed by the SEQ.

• Unless there is a compelling reason to use one of the other questionnaires, our recommendation is to use the SUS for post-study and SEQ or SMEQ for post-task assessments.

• Due to their growing use for commercial transactions, standardized usability questionnaires for websites include items focused on the assessment of attributes such as trust and service quality.

• Recent research indicates that the common practice of mixing the tone (positive and negative) of items in standardized usability questionnaires is more likely to harm rather than benefit the quality of measurement.

• Recent research also indicates that minor adjustments to the wording of items in standardized usability questionnaires do not appear to have an effect on the resulting scores (but extreme changes can affect the resulting metrics).

• The scores from standardized usability measurements do not have any inherent meaning, but they are useful for comparisons—either between products or conditions in usability studies or against normative databases.

• Commercial usability questionnaires that provide comparison with normative databases are the SUMI, WAMMI, and SUPR-Q.

• For noncommercial usability questionnaires, some normative information in the public domain is available for the PSSUQ and CSUQ (Lewis, 2002) and researchers have recently published norms for the SUS (Bangor et al., 2008; 2009; Sauro, 2011a).

• Questionnaires from the market research literature that may be of interest to usability practitioners are the ASCI, NPS, CxPi, and TAM scales (Perceived Usefulness and Perceived Ease-of-Use).

Chapter review questions

1. You’ve run a study using the PSSUQ (standard Version 3), with the results shown in Table 8.10. What are each participant’s overall and subscale scores, and what are the mean overall and subscale scores for the study?

2. Given the published information about normative patterns in responses to the PSSUQ, are you surprised by the mean score of Item 7 relative to the other items for the data in Table 8.10? What about the relative values of InfoQual and IntQual? Based on the typical values for the PSSUQ, does this product seem to be above or below average in perceived usability?

3. Suppose you’ve run a study using the standard version of the SUS, with the following results (Table 8.11). What are the SUS scores for each participant and their average for the product?

4. Given the published information about typical SUS scores, is the average SUS for the data in Table 8.11 generally above or below average? What grade would it receive using the Sauro-Lewis SUS grading curve? If you computed a 90% confidence interval, what would the grade range be? If these participants also responded to the NPS Likelihood to Recommend item, are any of them likely to be Promoters? Using those estimated Likelihood to Recommend ratings, what is the estimated NPS?

Table 8.10

Sample PSSUQ Data for Review Question 1

	Participant
	1	2	3	4	5	6
Item 1	1	2	2	2	5	1
Item 2	1	2	2	1	5	1
Item 3	1	2	3	1	4	1
Item 4	1	1	2	1	4	1
Item 5	1	1	2	1	5	1
Item 6	1	1	4	1	4	3
Item 7	1	2	.	1	6	1
Item 8	3	1	.	1	6	1
Item 9	3	1	1	1	5	1
Item 10	1	3	2	1	4	1
Item 11	2	2	2	1	4	1
Item 12	1	1	2	1	4	1
Item 13	1	1	2	2	4	1
Item 14	1	1	2	3	4	1
Item 15	1	1	3	1	4	1
Item 16	1	1	2	1	4	1

Table 8.11

Sample SUS Data for Review Question 3

	Participant
	1	2	3	4	5
Item 1	3	2	5	4	5
Item 2	1	1	2	2	1
Item 3	4	4	4	5	5
Item 4	1	3	1	1	1
Item 5	3	4	4	4	5
Item 6	1	2	2	2	1
Item 7	4	3	4	3	5
Item 8	1	2	1	1	1
Item 9	4	4	5	3	5
Item 10	2	1	2	3	1

Answers to chapter review questions

1. Table 8.12 shows the overall and subscale PSSUQ scores for each participant and the mean overall and subscale scores for the study (averaged across participants). Even though there is some missing data, only two cells in the table are empty, so it’s OK to just average the available data.

2. To answer these questions, refer to Table 8.2 (PSSUQ Version 3 norms). Regarding Item 7, generally, its scores tend to be higher (poorer) than those for other items, but in this set of data, the mean item scores are fairly uniform, ranging from 1.67 to 2.40, with 2.20 for Item 7, making it one of the higher scoring items but not as much higher as is usual, which is a bit surprising. The same is true for the relative pattern of the subscales. Typically, InfoQual is about half a point higher than IntQual, but for these data the difference is only about 0.15. Based on the typical values for the PSSUQ, the mean Overall score is usually about 2.82, so with an Overall score of 1.98, this product seems to be above average in perceived usability—at least, in reference to the products evaluated to produce the norms in Table 8.2. To determine if it is significantly better than average, you’d need to compute a confidence interval on the data from the study to see if the interval included or excluded the benchmark. It turns out that the 95% confidence interval for Overall ranges from 0.622 to 3.33—a fairly wide interval due to the small sample size and relatively high variability—so even though the mean is lower than the norm, the interval is consistent with the norm. This Overall score is not statistically significantly different from the norm of 2.82.

3. Table 8.13 shows the recoded item values and SUS scores for each participant and the mean SUS score for the study averaged across participants.

4. Based on the data collected by Sauro (2011a), the mean SUS score across a large number of usability studies is 68, so the mean from this study is above average. On the Sauro-Lewis SUS grading curve, scores between 80.8 and 84.0 get an A (Table 8.5). A 90% confidence interval on these data ranges from about 71 to 93, so the corresponding grade range is from C to A+ (at least you know it’s probably not a D), and because the confidence interval does not include 68, the result is significantly above average (p < 0.10). If these participants also responded to the NPS Likelihood to Recommend item, only one of them is likely to be a Promoter (responding with a 9 or 10 to the Likelihood to Recommend question). The simplified regression equation for estimating Likelihood to Recommend from SUS is LTR = SUS/10, so the predicted Likelihood to Recommend responses for these five participants are, respectively, 8, 7, 8, 7, and 10. Given these LTR scores, there are 0% Detractors and 20% (1/5) Promoters, for an estimated NPS of 20% (%Promoters minus %Detractors).

Table 8.12

Answers for Review Question 1

	Participant
	1	2	3	4	5	6	Mean
Item 1	1	2	2	2	5	1	2.17
Item 2	1	2	2	1	5	1	2.00
Item 3	1	2	3	1	4	1	2.00
Item 4	1	1	2	1	4	1	1.67
Item 5	1	1	2	1	5	1	1.83
Item 6	1	1	4	1	4	3	2.33
Item 7	1	2	.	1	6	1	2.20
Item 8	3	1	.	1	6	1	2.40
Item 9	3	1	1	1	5	1	2.00
Item 10	1	3	2	1	4	1	2.00
Item 11	2	2	2	1	4	1	2.00
Item 12	1	1	2	1	4	1	1.67
Item 13	1	1	2	2	4	1	1.83
Item 14	1	1	2	3	4	1	2.00
Item 15	1	1	3	1	4	1	1.83
Item 16	1	1	2	1	4	1	1.67
Overall	1.31	1.44	2.21	1.25	4.50	1.13	1.97
SysUse	1.00	1.50	2.50	1.17	4.50	1.33	2.00
InfoQual	1.83	1.67	1.75	1.00	4.83	1.00	2.01
IntQual	1.00	1.00	2.33	2.00	4.00	1.00	1.89

Table 8.13

Answers for Review Question 3

	Participants
	1	2	3	4	5
Item 1	2	1	4	3	4
Item 2	4	4	3	3	4
Item 3	3	3	3	4	4
Item 4	4	2	4	4	4
Item 5	2	3	3	3	4
Item 6	4	3	3	3	4
Item 7	3	2	3	2	4
Item 8	4	3	4	4	4
Item 9	3	3	4	2	4
Item 10	3	4	3	2	4
						Mean
Overall	80	70	85	75	100	82.00
Pred-LTR	8	7	8	7	10	Grade: A

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Key points

Create new playlist

Sign In

Sign Up

Key points

Chapter review questions

Answers to chapter review questions

Table of Contents for
Key points