Training record-keeping
Abstract:
This chapter examines training record-keeping in the life sciences and other regulated industries. Record-keeping is necessary for any training system that is subject to audit. This necessity may come to be recognized as early as the point in program development when training assessments are created. It certainly comes to be recognized when the program is being implemented (or even piloted) and actual training records are generated. This documentation could include training records (or attendance sheets), training assessments, and curricula or individual training plans (ITPs). These documents could be in electronic form or hard copy. This chapter will first consider some strategic aspects of record-keeping, then turns to tactical issues.
11.1 Introduction
In the mid-1990s, the University of Pittsburgh conducted a major study of functional requirements for record-keeping, called the Electronic Records Project.1 Reporting on this project, and specifically describing records, Wendy Duff stated: “[Records] are created in the first instance to control or direct an organization and to help orient staff to a common goal or purpose.”2 That is, records serve the purpose of controlling and directing the organization. She continues:
They have residual value because they document the outcomes of the directing and controlling activities and because they provide evidence of an organization’s rights as well as its obligations to its staff and society. For records to fulfill these roles, they must be readable, understandable, and trustworthy.3
There are two main audiences for record-keeping: operational staff and various quality auditors. The operational perspective is typically proactive, while the auditor’s perspective is typically retroactive. There are also other audiences, including the training unit itself.
Operational staff includes employees (the trainees) and their supervisors. Both employees and supervisors are interested in the trainees’ currency in their individual training plans (ITPs), for purposes of work assignments. At the beginning of each shift, the supervisor wants to know if the employees on this shift are trained to the current versions of each and every standard operating procedure (SOP) that is listed in the ITP, that will be executed during that shift. The supervisor reviews the employees’ training histories (i.e., the summary of the training records). Then the supervisor makes work assignments accordingly. Thus, the training records are used proactively to control and direct the organization.
Auditors include internal and external auditors (e.g., regulatory investigators, etc.) who are interested in whether the signer of the particular operational document (e.g., a batch record) was trained to the appropriate SOP before signing. The auditor reviews the signer’s training history in light of a set of documents being inspected. In these cases, the training records provide evidence of the organization’s past fulfillment of its regulatory obligations.
11.2 Record-keeping requirements
As the Electronic Records Project at the University of Pittsburgh has indicated, record-keeping requirements can be considered at several levels – that of the organization, that of the record-keeping system, and that of the record itself (Figure 11.1).
To begin with the highest level of requirements, the organization (i.e., Level I) must be compliant with all relevant legislation, regulations, and best practices concerning training records.4
At the second level of requirements, the training recordkeeping system (Level II) – whether electronic, paper based, or hybrid – must be implemented, responsible, consistent, and appropriately backed up according to the following definitions:
Implemented means that training events can be duly recorded in the system.
A responsible system’s controlled documents (i.e., SOPs for training record-keeping) are written and followed, plus the procedure clearly identifies the responsible party for each task.5 As an example of the failure to meet this functional requirement, and its GXP implications, consider FDA Warning Letter to Arrow International, dated 10 October 2007: “According to procedure #CHR-001. ‘Training Administration, Documentation, and Record-keeping Procedure,’ your firm has 30 days to complete the training. One training requirement was over six months late.”6
Consistent systems have been validated, so identical processes generate identical outcomes. Vendors sometimes suggest that their software has been validated or audited. However, FDA has specifically stated that the organization using the software must validate it for that situation.7
Appropriately backed-up systems protect documents from loss or corruption by being subject to a regularly scheduled backup. As an example of the failure to meet this functional requirement, see FDA Warning Letter to the Cordis Corporation in Warren, NJ, dated 1 April 2004: “… validation did not include testing and verification of back-up and restoration of the electronic data files.”8
The documentation of the training itself can be viewed as a captured record, a maintained record, and a usable record.
The required characteristics of a captured training record are authorized, comprehensive, identifiable, and complete according to the following definitions:
Authorized training records have been created by an authorized person, for example a qualified trainer.
Comprehensive means that a training record has been created for every training event. For an instance of not meeting this functional requirement, see FDA Warning Letter to Rhytec, Inc., dated 24 April 2007: “Documentation of training is not consistently maintained.”9
Identifiable means only one training record has been created for a given training event, and it is linked to that particular training event.
Complete training records include all information about the training event, for instance, which employee was trained, on which SOP, by whom (the trainer), and at what time and date. As an instance of the failure to meet this functional requirement, consider FDA Warning Letter to Omnicare, Inc., dated 11 January 2007: “all of the employee records lacked the ‘Supervisor Signature’ to show that the training was given.”10
Maintained records must be inviolate, auditable, and appropriately retained according to the following definitions:
Inviolate is defined as any alteration or modification of the record is traceable, and further, that repudiation of the record is not possible. As an illustration of not meeting this functional requirement, consider FDA Warning Letter to Concord Laboratories, dated 11 July 2006: “Appropriate controls are not exercised over computers or related systems to assure that changes in analytical methods or other control records are instituted only by authorized personnel.”11
Auditable means that every use of the record leaves an audit trail. As an example of the failure to meet this requirement, see FDA Warning Letter to Concord Laboratories, dated 11 July 2006: “… review of audit trails is not required.”12
Appropriately retained training records must be subject to a retention schedule and then disposed according to procedure.13
Usable training records must be exportable, retrievable, and accessible to authorized parties according to the following definitions:
Exportable records must be portable from one system to another without loss of information.
Retrievable training records are in a form that can be searched and retrieved within a reasonable period of time and expenditure of resources.
Documents accessible to authorized parties must be available to those who are authorized to access them and unavailable to those who are not authorized.14
After identifying two main audiences for the documentation of training – operational staff and auditors – the training recordkeeping must possess characteristics of good documentation management. If at each level – organization, training recordkeeping system, and documentation of training – characteristics are present that are appropriate for that level and proceduralized, that level will be “audit proof,” which is to say it can survive an internal or external GXP audit, and will moreover have business value to operational staff.
11.3 Part 11 compliance
When document management is discussed with reference to training and assessment, the topic of Part 11 compliance frequently comes up (Part 11 refers to “Electronic Records; Electronic Signatures,” which is Part 11 of 21 CFR). In keeping with the emergence of electronic technologies, FDA issued regulations in 1997 for e-records and e-signatures that sought to permit wide use of electronic technology, compatible with the protection of public health. Soon after they became effective, FDA announced a reconsideration of these regulations. In 2003, FDA withdrew the guidances that had accompanied the regulations. While the reconsideration of the regulations was under way, FDA indicated they would narrowly interpret the scope of Part 11 and promised to exercise enforcement discretion. During this period, records and record-keeping need still comply with the underlying regulations.
A typical example of FDA regulations and associated record-keeping is quality complaints about regulated products. 21 CFR 211.204 requires written procedures for the handling of all product quality complaints. This requirement (“predicate rule”) further stipulates that “a written record of each complaint shall be maintained in a file designated for drug product complaints.”
That is a second predicate rule; since it deals with recordkeeping, it implicates Part 11, if the organization has chosen to manage that record electronically. Moreover, the initial regulation also stipulates that a record shall be maintained, should an investigation of the product complaint be conducted; or the record shall include the reason and the name of the person responsible for a decision not to conduct an investigation. That is a third predicate rule; since it also deals with maintaining records of investigations, it also implicates Part 11.15
Equipment cleaning and maintenance under good laboratory practice (GLP), good manufacturing practice (GMP), and medical device regulations have broader scope (Table 11.1). The cleaning and maintenance requirement (first predicate rule) also stipulates that the cleaning and maintenance must be recorded (second predicate rule).16
Table 11.1
Equipment cleaning and maintenance under GLP, GMP, and medical device regulations
| Regulation | First Predicate Rule | Second Predicate Rule |
| 21 CFR 58.63 | (a) Equipment shall be adequately inspected, cleaned, and maintained | (c) Written records shall be maintained of all inspection, maintenance, testing, calibrating, and/or standardizing operations |
| 21 CFR 211.67 | (a) Equipment and utensils shall be cleaned, maintained, and sanitized at appropriate intervals to prevent malfunctions or contamination that would alter the safety, integrity, strength, purity, and quality (SISPQ) of the drug product beyond the official or other established requirements | (b) Written procedures shall be established and followed for cleaning and maintenance of equipment, including utensils, used in the manufacture, processing, packing, or holding of a drug product |
| 21 CFR 211.182 | Equipment and utensils shall be cleaned, maintained, and sanitized at appropriate intervals to prevent malfunctions or contamination that would alter the safety, integrity, strength, purity, and quality (SISPQ) of the drug product beyond the official or other established requirements | A written record of major equipment cleaning, maintenance (except routine maintenance such as lubrication and adjustments), and use shall be included in individual equipment logs that show the date, time, product, and lot number of each batch processed |
| 21 CFR 820.70 | (g) Each manufacturer shall ensure that all equipment used in the manufacturing process meets specified requirements and is appropriately designed, constructed, placed, and installed to facilitate maintenance, adjustment, cleaning, and use | (1) Maintenance activities, including the date and individual(s) performing the maintenance activities, shall be documented |
The form of these typical regulations involves two aspects: a requirement (one predicate rule) that a task or activity be proceduralized and the SOP be followed, and a requirement (a second predicate rule) that an associated record be kept of the activity or task. The second predicate rule, dealing with record-keeping, implicates Part 11 if the organization had decided to manage the record electronically. Insofar as Part 11 is implicated, procedures and controls must ensure the authenticity and integrity of electronic records. Moreover, procedures and controls must hold individuals accountable and responsible for actions initiated under their signatures.
11.3.1 Training records
By contrast, the documentation of training, including training records and training assessments, is not covered by such predicate rules. FDA regulations for areas such as pharmaceutical and biopharmaceutical operations, clinical trials, medical device operations, or human tissue processors require that personnel be trained. 17 These are examples of the first predicate rule noted in Table 11.1.
The only requirement for documentation of training is found in FDA GLPs, where it is stipulated that “Each testing facility shall maintain a current summary of training and experience and job description for each individual engaged in or supervising the conduct of a nonclinical laboratory study.”18 That implicates a “current summary” of the individual’s training records, which might take the form of the individual’s training history, not the training records or training assessments themselves.
Regarding clinical trials, FDA stipulates that:
A protocol is required to contain the following […] The name and address and a statement of the qualifications (curriculum vitae or other statement of qualifications) of each investigator, and the name of each sub-investigator (e.g., research fellow, resident) working under the supervision of the investigator
on a given clinical study.19 Notice that this predicate rule about “curriculum vitae or other statement of qualifications” of the clinical trials investigator was not extended to the subordinates, the research fellows, residents, etc.
Finally, the GMPs require “records shall be maintained stating the name, address, and qualifications of any consultants and the type of service they provide.”20 It is instructive that the rule-making that applied this predicate rule to the qualifications of consultants did not apply it to the majority of pharmaceutical operations employees and supervisors covered in 21 CFR 211.25.
FDA regulations are silent about training records for other areas such as pharmaceutical and biopharmaceutical operations, medical device operations, blood products processors, or human tissue processors.21 Indeed, CFR Part 11 does not include such a requirement for itself. It requires only a “determination that persons who develop, maintain, or use electronic record/electronic signature systems have the education, training, and experience to perform their assigned tasks,” which is another example of the first predicate rule noted in Table 11.1.22
In light of this, the documentation of training does not fall within the scope of Part 11. What are the implications of this limited scope? We assume throughout that this documentation is controlled, as well as duly signed by the party responsible for the action described. The documentation of training can be considered as instances of what FDA has called a hybrid situation. In such a situation, paper record and signature components can co-exist with electronic record and signature components, “so long as […] the content and meaning of those records are preserved.”23
Consider the following scenario: A GXP training event – either technical training or regulatory training – has just occurred. All the trainees have been assessed as “successful” in the training. There is a training record – a controlled document – and its use is proceduralized, including entry into a validated Learning Management System (LMS). Trainees then sign and date the paper-training record, and the trainer countersigns and dates the record. At this point, the event is fully documented; the trainees are fully trained to perform the GXP tasks. They can “touch” the product or supervise those touching the product. Then, according to procedure, duly authorized data entry clerks enter the data from the training record into the LMS within 72 hours, and sign off the entries. A duly authorized data steward verifies the data entries and signs off. At this point, by procedure, the electronic record becomes the controlled document, and the paper copy can be archived or disposed of.
Sounds straightforward; however, there have been situations where it is assumed that all training records fall within the scope of Part 11. Instead of the scenario outlined in the previous paragraph, the documentation of the training event is treated like a regulatory submission, say, where each of the parties involved must provide an electronic signature to the electronic record. So the “draft” training record is routed to each trainee for their review and electronic signature, then is routed back to the trainer for review and electronic signature, and finally routed to quality assurance (QA) for “release.” The number of “transactions” increases dramatically. When finally released, the training record ceases to be a draft, and becomes “effective.” Before the training record became “effective,” employees who had just been trained were instructed not to “touch” the regulated product.
In a study of a random sample of training records (n = 11) processed this way, involving employees – not consultants, non-clinical lab staff, nor clinical trials investigators – the average time between the conclusion of the training event and the “release date” was 10 days.
If training records had been recognized as outside the scope of Part 11 and the first scenario had been followed, the time between conclusion of the training event and the full documentation of the training would have been about 10 minutes – the time it takes the trainer to check all the trainees’ signatures and dates and countersign the training record.
FDA regulations require that personnel touching the product be trained. The regulations, with few exceptions, do not address training records, including training assessments. It is important to review carefully the cases where the regulations do apply and ensure compliance. It is equally important to ensure that the organization is not wasting time and resources in overbuilding (i.e., hyper-scoping the training record-keeping process).24
11.4 Tactics of training record-keeping
Training record-keeping includes training records (or attendance sheets), training assessments, and curricula or ITPs. In each case, this is a controlled document; each has necessary fields and optional (“nice-to-have”) fields. An example of a “nice-to-have” feature of a training record would be a field where a supervisor vouches for the record creator’s qualification.
Training records, for instance, have a number of necessary fields. These fields are listed in Table 11.2, insofar as the documentation corresponds to the first scenario.
Table 11.2
Necessary fields in training records
| Field | Comment | |
| 1. | Employee (Trainee) Name | This name must correspond to the employee’s name as entered in the annually renewed signature card |
| 2. | Employee ID Number | Temporary employees, etc. who do not have an authenticated network login must be provided with a unique ID number |
| 3. | Course/SOP Name | What is the organization’s naming convention? |
| 4. | Course/SOP Number | What is the organization’s numbering convention? |
| 5. | Type of Training | This includes one of a number of delivery modalities – classroom, e-learning, coaching, etc. |
| 6. | Date of Training Event | What is the organization’s date/time convention? |
| 7. | Trainer’s Name | This name must correspond to the trainer’s employee name as entered in the annually renewed signature card |
| 8. | Trainer’s Employee ID Number | Consultants, etc. who do not have an authenticated network login must be provided with a unique ID number |
| 9. | Data Clerk Name | [See comment to # 1 above] |
| 10. | Data Clerk’s Employee ID Number | [See comment to # 2 above] |
| 11. | Date Entered into LMS | [See comment to # 6 above] |
| 12. | Data Steward Name | [See comment to # 1 above] |
| 13. | Data Steward’s Employee ID Number | [See comment to # 2 above] |
| 14. | Date Verified | [See comment to # 6 above] |
The training procedure must not only list each of these fields, plus any optional “nice-to-have” fields, but must indicate the roles and responsible party for each field. If the training SOP describes a fully electronic scenario where trainees record their participation in the training event online, and trainers also record their facilitation of the event online, the SOP must also describe the fall back process in the event there are connectivity problems, training in remote locations, etc. Thus the roles and responsibilities of the data clerk and data steward are still critical.
If hard copies are archived following data entry into the validated LMS, they should be placed in the document repository with, at a minimum, the following indexing information: record type, file names, date ranges, and positions (titles of personnel), who are authorized to access the archived records. Or, by procedure, the hard copies should be appropriately disposed.
11.5 The training unit as audience
The training unit itself is another audience for the documentation of training. First, the training unit can use training records to document the level of effort of the unit. This documentation can be used in annual budget sessions to justify requests for more training staff or other resources.
Second, the training unit can use training records and training assessments to test the accuracy of statements about training.
Documentation of training impact can be represented as a continuum ranging from an endpoint reflecting training inputs to an endpoint reflecting training outputs (Figure 11.2).
At the first endpoint on the continuum, training impact is documented by staffing levels of the training unit. The supposition is, “The larger the training staff, the greater the training impact.” This is clearly only a measure of inputs, a very rough proxy for training impact.
At the other end of the continuum, training impact is documented by return on investment (ROI) for training. This can be calculated in a number of ways. For instance, the marginal benefit/cost ratio is the change in trainee proficiency divided by the change in the training unit’s expenditure on this training module.25 ROI is clearly a direct and salient measure of training outputs. Effective training leads to improved performance, hence performance measures should be used to document training impact. These measures might include numbers of documents with defects, amount of rework, and other business metrics.
The number of seats occupied in training sessions falls on that continuum, somewhere between staffing levels and ROI, nearer to the rudimentary endpoint of staffing levels.26 Other points on this continuum of the documentation of training impact include training assessments such as knowledge transfer assessments (KTAs) and skill demonstration assessments (SDAs).27
How can training documentation be used to test the accuracy of theoretical statements about training? Consider the following statements and accompanying graphic by Harold Stolovitch: “Because training is costly and removes workers from productive tasks, we expect significantly improved performance after training. Reality, however, is usually very different, as illustrated by the post-training solid line” (Figure 11.3).28 These statements and the graphic are quite intriguing to trainers.
Two methodological questions about Stolovitch’s figure arise: What kind of evidence would be required to determine whether there is a dip in performance during the training event (the second panel in the triptych)? What kind of evidence would be required to determine if there is a dip in performance following training (the third panel in the triptych)?
This chapter stresses the methodological point that the answer to each question depends on continuous tracking of performance – throughout the training event for the first question, and during the immediate post-training period for the second. Such tracking, taking the form of performance assessments (most likely in the form of SDAs), will require a substantial record-keeping effort.
In the typical case, however, assessments are conducted at only two points in time – one a pre-training assessment just before the training event, and a second, post-training assessment, say an “intermediate test” conducted after the trainees have returned to the job. Given only two data points, the average performance level would appear to be a straight line, perhaps upward-sloping to the right, perhaps flat; in any case we would not be able to address the questions about Stolovitch’s figure.29
These uses of the documentation of training do not have the enterprise-wide significance of the other two audiences– the operational use and the auditor’s use. The operational staff represents the line of business. This audience and its proactive use of training records for work assignments directly relate to the bottom line. The auditors represent the regulatory requirements within which the organization will be profitable (or not).
The training unit is usually viewed as an overhead department, engaging in capacity building and thereby (only indirectly) contributing to the bottom line. Donald Kirkpatrick and others have held that “trainers must justify their existence.”30 An effective training department should address business performance issues, and “justify their existence” by pointing to the improved performance of the workforce. Otherwise the training department will be viewed as overhead and not active contributors. In this sense, trainers are indeed an audience for training record-keeping. However, we see that this is a distinctly secondary audience, behind the two major audiences.
Good training record-keeping may well contribute to a corporate climate that supports widespread and disciplined use of organizational metrics. Such metrics allow benchmarking and trending to enhance organizational control.
11.6 Conclusion
There are two main audiences for training records, operational staff and auditors. In addition, there are other audiences such as the training unit itself. To serve these audiences, training record-keeping must possess characteristics of good document management. At each level of the organization, document management must be appropriate so that training record-keeping will be “audit proof,” and will, moreover, have business value to operational staff.
Regulatory compliance, as it relates to training record-keeping, requires that personnel touching the product be trained. FDA regulations, with few exceptions, do not address training records. It is important to review carefully the several cases where the regulations do apply and ensure compliance. It is equally important to ensure that the organization is not wasting time and resources overbuilding the training record-keeping process.
The fields that are necessary for training records, as well as necessary roles and responsibilities, need to describe the fallback process in case there are access or other system problems in a fully electronic approach to training record-keeping. Validated electronic training tracking systems should be employed to manage training records and training assessments in an effective manner.
Training records can provide data to justify budget requests. They can provide data to test the accuracy of statements about training. The training unit’s use of these records will not have enterprise-wide significance; yet, such use can contribute to the overall impact of organizational metrics.
11.8 References
Bearman, D. Electronic Evidence: Strategies for Managing Records in Contemporary Organizations. Pittsburgh, PA: Archives and Museum Informatics; 1994.
Duff, W. Ensuring the preservation of reliable evidence. Archivaria.. 1995; 42:28–45.
Engebretson, J. Federal agencies must begin to support new standard for logical access this month. SDM: Sec. Dist. Market.. 2006; 36(8):61.
FDA. Guidance for Industry: Computerized Systems Used in Clinical Investigations. Washington, DC: Office of the Commissioner, FDA, May; 2007.
Fischer, L. Condition critical: developing records retention schedules. Info. Man. J.. 2006; 40(1):26–34.
Hedstrom, M. Electronic Records Management Program Strategies. Pittsburgh, PA: Archives and Museum Informatics; 1993.
Kirkpatrick, D.L. Evaluating Training Programs. San Francisco: Berrett-Koehler; 1994. [p. 18].
Matus, R. It is no longer business as usual when it comes to handling electronic documents. J. Health Care Comp.. 2007; 9(2):11–73.
McDowell, R. Quo Vadis: 21 CFR 11. LC • GC Europe [Liquid Chromatography – Gas Chromatography]. 2004; 17(2):80–81.
Moore, C. Learning from key learning indicators. Chief Learn. Off.. 2007; 6(8):39.
Myler, E. The ABCs of records retention schedule development. E-DOC Mag.. 2006; 20(3):52–56.
Phillips, J. Measuring the ROI of a coaching intervention. Part 2. Perf. Imp.. 2007; 46(10):10–23.
Phillips, J., Phillips, P. Show me the money: the use of ROI in performance improvement. Part 1. Perf. Imp. 46(9). (October):2007.
Riordan, D. How enterprise software facilitates FDA compliance. Qual. Dig. 27(12). (December):2007.
Schneier, B. Two-factor authentication. Comm. ACM. 2005; 48(4):136.
Shukla, R. The case for electronic records management. Fin. Exec.. 2004; 20(7):50–52.
Stephenson, D. Accelerating compliance. E-DOC Mag.. 2007; 21(4):27–29.
Stolovitch, H.D. The story of training transfer. Talent Man. Mag.. 2007; 3(9):12.
Tennant, C., Boonkrong, M., Roberts, P. The design of a training programme measurement model. J. Europ. Ind. Trg.. 2002; 26(5):230–240.
Torres, T. Creating a process-focused retention schedule. Info. Man. J.. 2006; 40(5):62–69.
Wise-Blackman, G.M. Out-of-specification results and the quality system approach to GMPs. Pharma. Tech.. 30, 2006. [Supplement].
Woodrum, T. 21 CFR Part 11: The role of predicate regulations and associated internal policies. Drug Info. J.. 2003; 37(2):159–164.
1See Margaret Hedstrom (1993) and David Bearman (1994).
2See Wendy Duff (1995), esp. p. 29.
3See Duff, op. cit., p. 29
4See Duff, op. cit., pp. 33–5. These are functional requirements of record-keeping, not necessarily GXP regulatory requirements. As we shall see, the functional requirements do have GXP implications. For the current situation on organizational compliance, see Roger Matus (2007) and Darwin Stephenson (2007).
6Available at www.fda.gov/foi/warning_letters/s6537c.htm See also “Arrow warned about quality systems,” Reading Eagle, 16 October 2007.
7Federal Register, Vol. 62, No. 54 (20 March 1997), “Rules and Regulations,” p. 13445. See also Gwendolyn M. Wise-Blackman (2006): “validation must be achievable with staff at the work site.”
8Available at www.fda.gov/foi/warning_letters/archive/g4601d.htm See also “Federal Regulators Find Fault with Stent-Making Practices at Florida’s Cordis,” Miami Herald, 6 April 2004.
9Available from: www.fda.gov/foi/warning_letters/s6341c.htm
10Available from: www.fda.gov/foi/warning_letters/g6208d.htm
11Available from: www.fda.gov/foi/warning_letters/archive/g5973d.htm
12Available from: www.fda.gov/foi/warning_letters/archive/g5973dnhtm
13See Laurie Fischer (2006); also Ellie Myler (2006) and Tina Torres (2006).
14Authorization involves authenticated users; usually this is two-factor authentication involving two of the threefactors: (a) What you know (e.g., a password), (b) What you have (e.g., a security swipe card), and (c) What you are (e.g., a biometric characteristic). See Joan Engebretson (2006); Bruce Schneier (2005).
15On predicate rules, see Tammala Woodrum (2003). See 21 CFR 211.198 (a) on the requirement of written SOPs, 211.198 (b) on the requirement of written records for each complaint, and 211.198 (b) (2) and (3) on the requirement of written records for each investigation or the decision not to investigate.
16For GLPs, see 21 CFR 58.63, “Maintenance and calibration of equipment;” for GMPs, see 21 CFR 211.67, “Equipment cleaning and maintenance;” also 21 CFR 211.182, “Equipment cleaning and use log;” and for medical devices, see 21 CFR 820.70, “Production and process controls.”
17For pharmaceutical employees, see 21 CFR 211.25; for biopharm personnel, 21 CFR 600.10; for non-clinical lab personnel, 21 CFR 58.29; for medical device personnel, 21 CFR 820.25; for human tissue recovery personnel, 21 CFR 1271.170.
18See 21 CFR 58.29 (b), “Personnel.” As Robert McDowell (2004) has put it “It appears that Part 11 would not apply to computerized systems holding GMP training records, in contrast to GLP systems holding similar records where the rules would apply.”
19See 21 CFR 312.23(a)(6)(iii)(b). A curriculum vitae is more a record of educational attainment than a training history. Since the protocol is part of the IND regulatory submission, it will implicate Part 11 on that ground.
20See 21 CFR 211.34, “Consultants.”
21But see Wise-Blackman, op. cit., p. S-10, who repeatedly refers to “software that is 21 CFR Part 11 compliant and houses a database of training records.” The software that she advocates is not justified in terms of predicate rules – as we have seen, there are not any – but as follows: “One benefit of compliant training software is the ease of routine scheduling of required training” and “Routine retraining can be accomplished efficiently through the use of group sessions or individual web-based compliant software.” Of course, “routine scheduling” and “routine retraining” can be more easily and efficiently accomplished with any validated training tracking system, regardless of its Part 11 compliance.
22See 21 CFR 11.10 (i); see however Office of the Commissioner, “Guidance for Industry: Computerized Systems Used in Clinical Investigations,” Washington, DC: FDA (May 2007), p. 7:
Those who use computerized systems must determine that individuals (e.g., employees, contractors) who develop, maintain, or use computerized systems have the education, training, and experience necessary to perform their assigned tasks … We recommend that computer education, training, and experience be documented.
Neither a guidance nor a recommendation constitutes a predicate rule.
23Office of Compliance, CDER, “Guidance for Industry; Part 11, Electronic Records; Electronic Signatures – Scope and Application,” Washington, DC: FDA (August 2003), note 8. On the choice between electronic, paper-based, and hybrid record-keeping systems, see Rakesh Shukla (2004). See also Dan Riordan (2007), who echoes the first predicate rules on page 33:
The FDA requires that medical device manufacturers and pharmaceutical companies give their employees adequate training in their job responsibilities, and in their roles in ensuring the quality of a company’s goods and services.
Riordan continues with a list of software functions:
In compliance software systems, users can create documentation for training requirements, job descriptions, qualification definitions, courses, work instructions, and many other criteria. System administrators can configure multiple levels of secure access, so that all employee records remain confidential. Many systems have capabilities for managing and tracking employee certifications, scheduling courses and training sessions, and monitoring employee performance. Approved changes to documentation can automatically update employee training records and simultaneously notify appropriate employees and managers of the need for retraining on these documents. Automating these processes makes it much easier to keep employee records updated, and is an important element in FDA compliance.
24See Tammala Woodrum, op. cit., esp. pp. 162–3 on the problem of organizational “policies that potentially expand the scope of Part 11.” FDA has indicated that Part 11 was never intended to “significantly increase the costs of compliance to an extent that was not contemplated at the time the rule was drafted,” nor was it intended to “discourage innovation and technological advances without providing a significant public health benefit.” See Office of Compliance, CDER, op. cit. A hybrid record-keeping system might best address the situation where the vast majority of training documents would be maintained in electronic form, and the few exceptions would be managed in paper form.
25See Jack Phillips and P. Phillips (2007) and Jack Phillips (2007), esp. p. 18.
26Chris Moore (2007) refers to the number of seats occupied in training sessions as “fill rates.”
27It is not the case, contrary to Wise-Blackman, op. cit., p. S-10, that “documenting the transfer of knowledge about the SOP is best accomplished through a web-based system that incorporates short quizzes as a prerequisite to receiving approval for training,” because there is a substantial legal exposure to the use of unvalidated KTAs (short quizzes), and there are serious costs to validating KTAs. The trainee’s completion of prerequisites would best be ascertained through a SDA, as would the documentation of trainee proficiency.
28The quotation and figure are from Harold D. Stolovitch (2007).
29Charles Tennant, M. Boonkrong, and P. Roberts (2002) have suggested three kinds of post-training assessments – an “immediate test” when the training has been completed, an “intermediate test” when the trainee has returned to the job, and an “ultimate test” to measure behavioral changes.
30See Donald L. Kirkpatrick (1994). The best justification for trainers’ existence is probably record-keeping that satisfies the needs of operational staff and GXP auditors.