Appendix E

How to Measure the Measures

How well does your association perform on the seven measures?

This is an easy question to ask but a hard one to answer. For example:

• Many of the qualities that exemplify remarkable associations are intangible. The breadth and depth of a customer service culture (Measure 1) cannot be accurately measured by counting the number of “We love our members” plaques on the wall. Anyone can put up a plaque. Member focus is both an attitude and a set of practices.
• Several of the measures are not mutually exclusive. The presence of dialogue and engagement (Measure 4) and the model of the association CEO as broker of ideas (Measure 5) both reflect an organizational culture that values candor, where people from all quarters may speak freely without fear of retribution and innovate without being seen as disrespecting authority.
• Several measures are innately subjective and inevitably influenced by one’s role in the association. As an example, an association’s product development team may perceive its efforts as being closely aligned with the mission (Measure 2), while those engaged in product marketing may see things quite differently.

Fortunately, none of these issues presents an insurmountable obstacle to an association willing to embark on an objective measurement program. Doing so first requires a commitment to:

• Invest the time, money, and effort required to do measurement properly.
• Communicate the results broadly and with candor.
• Foster change in the areas identified as problematic by the research. Asking for feedback and then ignoring or discounting the messages received can diminish your organization’s credibility as well as its ability to gather honest feedback in the future.

Outlined below are several methods that you can use to ascertain how well your association is performing on each of the commitments and the criteria (measures) on which they are built. Some options are more ambitious; others offer smaller, less expensive methods for conducting measurements.

Commitment to Purpose

Measure 1: A Customer Service Culture

Both organizational policies and processes and unrehearsed interactions with individual members consistently express a “We’re here to serve you” attitude.

Potential Measurement Methods

• Develop a program to measure and analyze compliments and complaints. This is a simple and relatively inexpensive method of finding out what makes members feel valued and what engenders dissatisfaction.
• Administer the survey in the 7 Measures of Success Implementation Guide and Assessment Tool.
• Conduct focus groups, using an independent moderator, with front-line employees at your association. Include those who answer the phones, interact with members and/or volunteers at meetings, and so forth. Often, the people on the front lines know a great deal about members and the association’s attitudes toward them. Typically, front-line employees are also willing to speak candidly.

Measure 2: Alignment of Products and Services with Mission

The association’s mission remains consistent, and the association’s portfolio of products and services is congruent with its stated mission.

Potential Measurement Methods

• Identify the presence or absence of a written mission statement. Is there any evidence to suggest that the mission has played a meaningful role in shaping actions across time? Review minutes of meetings leading to and/or announcements of significant changes to see whether specific elements of the mission are mentioned.
• Look at major sources of income and areas of investment across a 10-year period. Do products and services that explicitly reflect the mission provide a major source of income? Has the association consistently invested time, money, and effort in the development of mission-consistent products and services?
• Conduct focus groups, using an independent moderator, organized around the topic: “What is our mission?” Hold focus groups with members, staff at all levels, and volunteers—but do not tell them in advance that you will be asking this question.

Commitment to Analysis and Feedback

Measure 3: Data-Driven Strategies

Remarkable associations have developed an expertise in gathering information as well as processes for sharing and analyzing the data to deduce what actions the data point to taking.

Potential Measurement Methods

• Count the number of channels through which you gather information, the types of information collected, and the places in your organization that engage in such activities. You may use informal channels, such as conversations with members or colleagues, and/or formal market research studies. You may seek both qualitative data (from focus groups, for example) and structured, quantitative data in the form of surveys; the data may be collected at numerous points throughout your organization, including human resources, member relations, customer service, and so forth. The more you find in each area, the better.
• Determine what happens to the information collected through the various methods. Does it ever move beyond the point of contact? To begin to tap its promise, information must be recorded, shared, and analyzed in relation to information from other sources.
• Next, ask “Does the information get communicated? If so, how is it communicated and to whom?” Information can be a powerful tool for framing and motivating action. Far from hurting your organization, transparency engenders trust, motivates change, and fosters innovation.
• Exercise discipline to collect accurate data. Specifically, record the information you collect in a standardized format and adhere to professional standards when conducting formal market research. Designing and administering a survey correctly requires skill and formal training; this responsibility should not be assigned to people who are not fully qualified. Remarkable associations believe in good, solid research and commit the resources necessary to do it well.

Measure 4: Dialogue and Engagement

An internal conversation continually occurs among staff and volunteers about the organization’s direction and priorities.

Potential Measurement Methods

• Conduct an employee survey, using either a standardized or customized survey program, to learn about the state of communication within your association. Ideally, seek the advice of an independent expert who has advanced training in psychometrics and the particular quirks of employee survey management. (Hint: Confidentiality issues loom large in employee surveys.) This expert should not have any institutional ties that could bias his or her advice, such as being employed by a company that sells survey instruments.
• Use 360° assessments of randomly selected employees, volunteers, staff, members, and suppliers as another means of discerning the state of communication within your association. Again, many such instruments are available; ideally, seek the advice of an independent expert before making your selection.

Measure 5: CEO as Broker of Ideas

Although the CEO may be visionary, what’s more important is that the CEO facilitates visionary thinking throughout the organization.

Potential Measurement Methods

• Review the tenure of current and past CEOs. Can you find examples of ideas, new products or services, or initiatives that were generated by someone else? If so, how many were there? Did any of these initiatives receive institutional support in the form of time, money, and/or publicity? If so, how did the level of support compare to initiatives that originated with the CEO? How many of these proposals were ultimately implemented?
• On a daily, weekly, or monthly basis, determine how much time the CEO spends—outside his or her office and/or the association’s headquarters—listening to and interacting with various stakeholders. How much time does the CEO spend learning about the world beyond the borders of the association? How often does he or she present truly new material, in any forum, for review and discussion? A simple count of such events can be highly instructive.

Commitment to Action

Measure 6: Organizational Adaptability

Remarkable associations learn from and respond to change; although willing to change, they also know what not to change.

Potential Measurement Methods

Answering the following questions can help you assess your association’s level of adaptability during times of change:

According to research by C.R. Wanberg and J.T. Banas reported in the Journal of Applied Psychology (“Predictors and Outcomes of Openness to Changes in a Reorganizing Workplace,” February 2000), resilient organizations do the following in times of crisis:

• Acknowledge that positive and negative changes exist, while projecting realistic optimism.
• Provide information generously, to all concerned.
• Invite participation on the part of all stakeholders in learning from and adapting to change, while simultaneously lending social and personal support to all affected.

Thus, answers to the four questions above are proxy measures for adaptability; the questions should be posed to all stakeholders.

Measure 7: Alliance Building

Associations that are secure and confident in their own right seek partners and projects that complement their mission and purpose.

Potential Measurement Methods

Measuring your association’s ability to build successful alliances goes beyond basic counting. Determine, for instance:

The answers to these questions will help an association begin to assess the extent to which any alliances were considered in terms of their alignment with mission and goals, core competencies, and the association’s willingness to invest in careful planning to enhance success.

Sir Francis Bacon noted that knowledge is power. Like any other powerful tool, the information embodied by the seven measures confers power on the user only if that information is collected carefully and ethically and used responsibly—in the service of the association’s stakeholders and its goals.