| 5.5 KPIs and Reporting |  |
Measure citizen development.
This section provides information about the metrics you need to track and manage citizen development projects and programs.
In this section you will find:
•A discussion of project performance metrics
•Guidance on how to measure citizen development progress
•A proposed series of KPIs to track citizen development metrics
“If you can't measure it, you can't improve it.”
Peter Drucker
Overview
This chapter presents the metrics that give visibility of the value being created by citizen development efforts. An example of this is tracking the number of shadow IT tools that have been added to the IT catalog.
We aim to provide a mechanism to track the effectiveness of citizen development project performance and the effectiveness of the project squad.
 | DID YOU KNOW? |
In 1998, a law passed in the U.S. state of Virginia allows drivers to keep their roadkill, as long as they report it within 12 hours (Fine Dictionary, n.d.).
KPIs and Reporting in Practice
A citizen development strategy requires collecting and tracking metrics that measure the benefits and effectiveness of the strategy. Depending on the maturity of citizen development within an organization, KPIs and reporting start to be reported in an ad hoc manner and then evolve to become more sophisticated as the organization matures in the deployment of citizen development. KPIs provide stakeholders in the organization with insights to the scale at which citizen development has been deployed. It will further enable leaders and managers to monitor the progress of citizen developer organizational maturity; measure the project squad; and measure practitioner effectiveness, organization adoption, and community engagement. It will also help answer frequently asked questions such as:
•Are we effectively allocating resources into citizen development?
•Is the organization accepting and benefiting from citizen development?
•Are individuals sufficiently trained and skilled to take part in citizen development?
•Is citizen development promoting cross-departmental collaboration?
•Are we addressing the problem of shadow IT?
Measuring citizen development implementations
Measuring citizen development implementations provides data on project implementation performance and maturity. Naturally, different vendor platforms will have differing types of data, based on how they are constructed and how users interact with and build applications. However, the platform is an important vector through which numerous KPIs and measures can be captured and reported on, which will provide a transparent view on the effectiveness of the organization's citizen developers and practitioners. Straightforward measures, such as the number of developers or applications that have been built, can provide useful insights over time. More specific measures, such as the number of reusable application templates or prepackaged functional blocks, can provide an estimate of citizen developer competence and efficiency. Other easily captured measures could be how many application users there are, and how many times an application is updated or downloaded by other citizen developers.
Aggregated measures
Tracking and monitoring of citizen development needs to be enabled at the project level as well as the aggregate level, whether it is performed by a department or by the organization as a whole. This necessitates the provision of cascading, integrated dashboards that provide dynamic aggregation of measures and KPIs.
However, this is much easier said than done. Enabling an effective performance measuring and tracking capability that spans the organization's levels requires the establishment of layers of governance and new accountabilities. As such, in the early stages of citizen developer maturity, measuring and tracking of citizen development will usually be managed through ad hoc reports. As citizen development matures and scales in the organization, this will need to become increasingly automated and interconnected.
 | DEFINITION |
Repository
A central location in which data is stored and managed (Google Dictionary, n.d.c.).
Metadata
A set of data that describes and gives information about other data (Google Dictionary, n.d.b.).
Measures enabled by the service catalog
The service catalog is a critical component of the citizen developer IT strategy. Capturing a single repository of all applications built through citizen development, combined with the descriptive metadata that facilitates simple and accurate search and identification based on keywords or hashtags, will be a powerful enabler. In some cases, the organization will have a citizen development platform where all the applications created are categorized and made available for viewing. This service catalog can be viewed directly when extracted from this citizen development platform. Regardless of whether this is captured within the platform or by a specific tool, the organization can start to measure and track stats such as how many times an application has been viewed, downloaded, used, or modified. Depending on the maturity of the service catalog capability, comments and interactions between the community and the developers can provide additional insights that could be tracked. Examples include community engagement based on numbers of comments, questions, and answers by the developers or users of the application, and perhaps a community-driven rating system.
Measuring the effect on shadow IT
Citizen development can bring control and governance by initiatives that were once performed as shadow IT, taking initiatives from sporadic, ad hoc, and unsanctioned development into a transparent, controlled, and compliant environment. This would necessitate the identification of the existing shadow IT landscape, which would be a largely manual effort, possibly through an extensive process of data gathering by interviewing team leaders across the organization and assessing the business process tools they use to get their work done.
Capturing a view on the shadow IT landscape and using this to populate a qualified pipeline of solutions suitable for citizen development is a great way to leverage business users and subject matter experts’ knowledge and experience, which would reduce operational risk and increase the governance and control from shadow IT application architecture. Measuring the progress of the conversion to citizen development initiatives will give stakeholders at every level of citizen development maturity the ability to see what lies ahead and how their progress is coming along.
Measuring citizen development delivery effectiveness
As with any change or IT project delivery, there are numerous metrics that should be captured across the life cycle of the project from ideation through to publishing and ongoing maintenance and upkeep. Combining measures at a “local level” with measures captured from the platform can provide rich insights into how well the citizen developers and supporting teams have collaborated to produce working solutions. For example, measuring the duration the project squad spends at each stage in the life cycle and combining that view with the scale, complexity, and number of users of the application can provide rich insight into competency development and velocity of application development for a given application.
Additionally, it would be possible to capture the number of citizen development applications that are built using the fast track versus those that are pushed through the IT delivery path. Typically, those projects going through the fast track path would be smaller applications involving a combination of relatively low-complexity solutions and suitably competent squad members. Those projects going through the IT delivery path would typically be larger endeavors involving a higher level of complexity and risk, which would progress through a path of increased governance. Measuring this would give a sense of the scale of applications being built using citizen development and can serve to give useful insights into the maturity of citizen development and the overall competence of citizen developers in the organization.
Measuring competency center effectiveness
In most organizations, building repeatable and sustainable practices will necessitate the creation of a citizen development competency center. In some organizations, similar outcomes can be achieved through an informal structure or community of practitioners. For organizations that are investing further in citizen development, the citizen developer competency center is typically set up during the adoption stage of organizational maturity and operation is scaled in concert with citizen development activity. Being able to track the value it brings to the organization will be a key driver for any investment decisions to scale it further.
The competency center will comprise a number of stakeholders and will represent citizen development best practices through the deployment of citizen development roles: the strategist, architect, practitioner, and community manager.
Table 19 serves as a guide of commonly used KPIs in citizen development.
