11.3.2.1 Frequency of maintenance actions

Once a maintainability block diagram is in place, projections about the number of maintenance actions, the times between maintenance actions, etc., may be made using the same techniques that were used in Chapters 3 and 4 for reliability block diagrams. The key is to prepare a block diagram that reflects the number of maintenance actions instead of the reliability of the system. As a rule, a system will undergo more maintenance actions than it will undergo failures, mainly because redundant units, while useful for preventing outages, may require attention when they fail in order not to leave the system in a brink-of-failure state.

This section discusses the use of the separate maintenance model (Section 4.4.5) as a model for the number of maintenance actions over a given time period. To implement the separate maintenance model in this application, begin with a maintenance functional decomposition (Section 11.3.1) in which every replaceable unit in the system is individually identified as an element of the decomposition. A maintenance functional decomposition is like a system functional decomposition, but it is constructed so that every maintenance action is recorded—even if it does not cause, or result from, a system failure. There may be other elements in the maintenance functional decomposition, but every subassembly or LRU that is designated as replaceable in the system maintenance plan should appear in the decomposition. A maintainability block diagram is a reliability block diagram based on the maintenance functional decomposition. Separate the diagram into two parts so that one part contains all the replaceable units. Let φ_M(X₁, . . ., X_n) denote the structure function of that part of the diagram containing the replaceable units (numbered 1, . . ., n).³ Finally, denote by Z₁(t), . . ., Z_n(t) the reliability processes (Section 4.3.2) of the n replaceable units.⁴ The separate maintenance model is the reliability process Z_M(t) = φ_M(Z₁(t), . . ., Z_n(t)) of the ensemble of the replaceable units. We use Z_M(t) to obtain the number of maintenance actions for the system.

Example: Continue the server rack example from Section 11.3.1. Let N₁(t), . . ., N₁₆(t) denote the number of unit failures in the time period [0, t] for units 1, . . ., 16, respectively. We consider two cases:

1. Each power supply module is replaced when it fails. In this case, the number of maintenance actions is N₁(t) + ⋅ ⋅ ⋅ + N₁₆(t) because every unit failure, including each power module, causes a maintenance action. When the operating time and outage time distributions for each of the system’s components are as shown in Table 4.1, the expected number of maintenance actions over 5 years is 49.363 if the units are replaced by new ones when they fail [21, 23]. If each unit is revived (Section 4.4.3.1) when it fails, the expected number of maintenance actions over 5 years is 79.579 (Exercise 2). The number of failures with revival is greater than the number of failures with renewal because of the increasing hazard rate nature of the life distributions for the server and the power supply.
2. When the power supply module in service fails, the backup power supply (if it is not already failed) is put into service; the ensemble of two hot standby power supply modules is replaced at the time of the second power module failure. In this case, the number of maintenance actions is N₁(t) + ⋅ ⋅ ⋅ + N₁₂(t) + N₁₅(t) + N₁₆(t) + J(t), where J(t) denotes the number of replacements of the ensemble of two hot standby power modules. The expected value of J(t) may be determined from equation (4.10) of Ref. 22 if an alternating renewal model for the modules is acceptable and the on- and off-time distributions in that model are assigned.

11.3.2.2 Duration of maintenance actions

Maintenance action durations of interest include

• duration of an individual operation at a single workstation and
• total time needed for a single maintenance job to transit a facility.

These may pertain to preventive or corrective maintenance.

The sojourn time of a job at a single workstation may be estimated from historical data or may be measured using a time-and-motion study [9] . In a maintainability context, these studies are also known as maintenance task analysis [3].

Information about the total time a job spends in a maintenance facility may be gained from a precedence diagram (critical path method) or activity network model [9] for the facility. For purposes of this analysis, a maintenance facility may be conceptualized as a network of workstations with jobs flowing around the network in a pattern determined by the type of equipment being serviced, its service needs, and the types of tasks that can be performed at each of the workstations in the facility. Discussion of this variable is postponed until Section 13.4.1 in which we describe a stochastic network flow model for performance analysis and optimization of a maintenance facility.

11.4.2.1 Online maintenance

Online maintenance refers to preventive or corrective maintenance actions that take place where the system is deployed. Usually, online maintenance comprehends simpler, shorter-duration, or less frequently–occurring tasks, such as cleaning, minor adjustments, periodic condition monitoring, and the like, that may be accomplished by system operators without taking the system out of service. Online maintenance may also be preferred for rapid replacement of critical items to minimize system outage time.

Maintenance training for system operators incorporates

• ability to initiate and interpret diagnostic routines for fault location,
• procedures for the maintenance tasks (preventive or corrective) that are determined by the LoRA to be performed on-site, and
• ability to discern when a repair is outside the scope of on-site maintenance but needs to be referred to a higher level for completion.

Planning for environmental influences on repair procedures is more important at the online level because systems may be deployed in a variety of differing environments: shipboard, aircraft, automotive, poor air quality, arctic or equatorial, etc.

11.4.2.2 Off-line maintenance on-site or at a nearby site

Certain preventive or corrective maintenance actions may require that a system be taken out of service before they can be performed. The system is said to be off line, and the maintenance may then be undertaken on site or at a nearby fixed or mobile location. This, and off-line maintenance at a remote location (Section 11.4.2.3), is referred to as “intermediate maintenance.” For example, some line-replaceable units (LRUs) may require the system to be powered down before they can be safely removed or replaced.

11.4.2.3 Off-line maintenance at a remote location

This is the second type of intermediate maintenance, sometimes called depot-level maintenance. It should be considered for repairs that

• may be more complicated,
• may require specialized test equipment and/or tools,
• may require more specialized expertise to accomplish, or
• may occur less frequently (so that on-site stocking of spare part(s) required for this repair may be costly).

Turnaround times will be greater than for online maintenance or for off-line maintenance on site or nearby, and transportation costs are also incurred.

This is also the first level of maintenance where it is reasonable to consider repair of LRUs. Many LRUs are valuable enough that they are not discarded when they fail but are instead repaired and placed into a spares inventory for use in future system corrective maintenance. Repair of an LRU usually entails replacement of some component(s) on the unit, so solder rework stations and other specialized tools may be required. This is also usually exacting work, and it is not reasonable to expect that it could be performed under field conditions (even an environmentally controlled location such as a telephone central office, while offering a benign (stable temperature and humidity, low vibration, etc.) environment, would not be equipped with the workspaces and workstations needed for these repairs). When a system contains valuable LRUs that are repaired and not discarded when they fail, depot- and/or manufacturer-level maintenance is almost mandatory. Figure 5.1 shows an example of the flows of material and information supporting a repair scheme in which LRUs are repaired at an independent facility under contract to the system manufacturer. That is, the system manufacturer has outsourced the repair of LRUs that it might have performed itself. Note that Figure 5.1 does not allude in any way to the potential political difficulties that may arise in seeking to share reliability data across unrelated organizations. While this is an important consideration, it is beyond the scope of this book.

11.4.2.4 Off-line maintenance at a manufacturer’s or supplier’s facility

For systems or products using a multilevel maintenance concept, this is the last resort for repairs. Consider reserving this level of maintenance for

• particularly intractable failures that are difficult to diagnose,
• situations where long turnaround times can be tolerated, or
• tasks that are beyond the capability of on-site or intermediate maintenance staff.

A multilevel maintenance scheme also offers the possibility of spilling over to the next higher level repairs that have been attempted but were unsuccessful. It is reasonable to expect (but should be verified) that the manufacturer of the system has the specialized expertise, tools, and diagnostic systems to handle almost every type of failure. For some types of products (e.g., consumer entertainment products), this may be the only option offered by the manufacturer (even though the owner may be able to perform repairs himself or may contract repair to be conducted by an independent shop).

As with intermediate maintenance, costs will be incurred for transportation of materials both to and from the facility.

11.4.2.5 Analysis and optimization

The LoRA described in this section helps choose the least cost repair scheme to fit the particular needs of your system. From the four levels of repair and the possibility of discarding the item, there are at most 31 combinations (ranging from using on-site repair only up to using all four levels plus discard). Some combinations may be ruled out by other conditions prevailing in system use or operation, so the number of choices is usually limited to a small number. Choice of the least cost option is readily accomplished through use of a spreadsheet-based accounting procedure. LoRA is described in detail in MIL-STD-1390D [24], which, while no longer supported by the Department of Defense, contains a wealth of information and procedures that help in practical LoRA studies. LoRA is also used in the automotive and aerospace industries [19] . Software to allow rapid completion of LoRA has been described [7, 11]. As with all off-the-shelf software, users should verify that the assumptions used by the software developers are appropriate for the study being undertaken before relying on the answers generated by the software.

To begin a LoRA, choose a time horizon for the decision process. Use a time horizon that reasonably reflects the time over which you expect that the system will be supported by the maintenance-level scheme. Use one spreadsheet worksheet for each item type to be maintained in the multilevel scheme. Use one column for each level-of-repair option (e.g., a three-level maintenance scheme uses four columns, one for each level and one for the “discard” option). Use one row for each of the following costs:

• Acquisition cost: the first cost of purchasing the item,
• Expected maintenance labor cost: labor cost per hour multiplied by the expected total duration of all maintenance tasks on that item over the chosen time horizon,
• Maintenance staff training cost allocated to the item,
• Maintenance facility costs (rent, utilities, janitorial costs, capital costs, etc.) allocated to the item (if necessary, separate capital costs from expenses),
• Inventory acquisition cost for the item,
• Inventory carrying cost for the item,
• Repair parts inventory acquisition cost for the item,
• Repair parts inventory carrying cost for the item,
• Cost of test equipment, tools, documentation, software, etc., for the item,
• Transportation costs for the item,
• Recycling and disposal costs allocated to the item.

Each cell is populated with the cost associated with its row for the level associated with its column. Use the spreadsheet to add up the costs down each column. The LoRA procedure chooses the option corresponding to the column with the lowest total. If you were making a decision for that one item only, you could do it now based on the spreadsheet results. If there is more than one item in the plan and all items are to be repaired using the same multilevel plan, make a weighted average of all the total costs, weighted by the proportion of the total population of items represented by each individual item. For instance, if there are two items A and B, and item A represents 30% of the total number of both items and item B represents 70% of the total, weight the total costs of items A and B by 0.3 and 0.7, respectively. Make separate worksheets for the two items and average the total costs over the two worksheets for A and B using these weights. The lowest weighted average total cost over the options considered is the LoRA solution. If there is more than one item in the plan but each item may have a separate repair scheme, the items may be treated individually with an optimal level of repair selected for each, and the weighted-average procedure is not needed.

The remainder of this section is devoted to a small example of a LoRA, not so much as a generic example to be followed to the letter in particular applications but more as an illustration of the procedure and the reasoning process that makes LoRA useful.

Example: This example concerns two LRUs, A and B, that are themselves repairable. The options considered include intermediate-level repair, depot-level repair of the LRUs (a failed LRU is replaced by a working one from a spares inventory so that the system is restored to service; the failed LRU undergoes repair itself using the scheme to be decided by the LoRA), and discard (when an LRU fails, it is not repaired but is discarded or recycled). The units are not repairable on-site. We will illustrate a LoRA for these units using a 10-year time horizon. To fill in the spreadsheet, some facts about units A and B are required.

1. A system containing units A and B is installed in 10 submarines. Each submarine contains two systems. Each system contains three A units and seven B units. The systems are in service 12 hours a day, and the submarines run on a 6-months-on, 2-months-off schedule, so over the 10-year study horizon, each system accrues a total of 32,400 hours of operation.⁵ The type A units accrue a total of 1,944,000 unit-hours, and the type B units accrue a total of 4,536,000 unit-hours.
2. Type A units cost $18,000 each if designed to be repairable and $15,000 each if designed to be discarded. The corresponding costs for unit B are $3500 and $2750, respectively.
3. The labor rate, including all overhead, is $35 per hour at the intermediate level and $55 per hour at the depot level. Unit A takes an average of 4 hours to repair, while unit B takes an average of 3 hours to repair.
4. The estimated failure intensity of unit A is 6 × 10⁻⁵ failures per hour and that of unit B is 2 × 10⁻⁵ failures per hour.⁶
5. Training repair personnel costs $60 per hour for intermediate-level staff and $80 per hour for depot-level staff.
6. Facility costs allocated to units A and B are $1.50 per maintenance hour at the intermediate repair level and $2.50 per maintenance hour at the depot repair level.
7. The spares inventory size is 2 type A spares and 5 type B spares per system at the intermediate level and 10 type A spares and 25 type B spares (covering all systems) at the depot level when the units are repaired. If the units are discarded, one spare is needed for every unit in the field. Inventory carrying costs are approximately 7% of the inventory value per year.
8. Parts consumed in the repair of unit A amount to $78 per repair and for unit B amount to $24 per repair.
9. Costs of test equipment, tools, etc., allocated to units A and B together are $12,000 per installation for intermediate-level repair and $50,000 per installation for depot-level repair.
10. Transportation costs $500 for any number of units from the submarine to either the intermediate or depot repair facility, regardless of the number of units being shipped.
11. Disposal costs $25 for unit A and $15 for unit B. Fifty percent of those disposed are recycled, and recycling unit A (B, respectively) brings in $80 ($10, respectively) in revenue.

The spreadsheet for unit A is shown in Table 11.1.

Cost Description	Intermediate Repair ($)	Depot Repair ($)	Discard Option	Notes
Acquisition	1,080,000	1,080,000	900,000	20 systems, 3 type A units per system
Labor	16,330	25,661	0	Expected number of failures over 10 years is 116.64
Training	11,520	2,880	0	Eight students for 3 days at intermediate, two students for 3 days at depot
Facility	700	1,166	0
Spares inventory acquisition	720,000	180,000	900,000
Spares inventory carrying	504,000	126,000	630,000
Repair parts consumption	9,098	9,098	0
Test equipment, tools, etc.	3,600	4,500	0
Transportation	0	0	0	Not included because it is the same in all scenarios
Recycling/disposal	0	0	−3,208
Total	2,345,248	1,429,305	2,426,792

The spreadsheet for unit B is shown in Table 11.2.

Cost Description	Intermediate Repair ($)	Depot Repair ($)	Discard Option	Notes
Labor	22,226	34,927	0	Expected number of failures over 10 years is 90.72
Training	11,520	2,880	0	Eight students for 3 days at intermediate, two students for 3 days at depot
Facility	408	680	0
Spares inventory acquisition	350,000	87,500	385,000
Spares inventory carrying	245,000	61,250	269,500
Repair parts consumption	2,177	2,177	0
Test equipment, tools, etc.	8,400	10,500	0
Transportation	0	0	0	Not included because it is the same regardless of the number of units
Recycling/disposal	0	0	227
Total	639,731	199,914	654,727

In both cases A and B, the analysis indicates that intermediate repair is preferred. Note that training costs are the same for both units A and B, so these could have been left out of the analysis, and the conclusion would not change. If we needed to consider both units A and B together (i.e., only a single level of repair strategy was to be selected for both units), the additional step of taking weighted averages of the results for A and B would be required if the two analyses pointed to two different choices for A and for B. In this example, this is not needed because the conclusion is the same for A and B: use intermediate-level repair.

This example is oversimplified and not intended to provide a template for any particular LoRA. Rather, it is intended to show in general terms how LoRA is carried out and what the underlying reasoning process is. Practical LoRA is facilitated by standards such as Refs. 19, 24 and off-the-shelf software such as Refs. 7, 11.

11.4.4.1 Predictive maintenance

There are two broad classes of RCM schemes: ones based on the knowledge of the pattern of system failures in time (or other age-measuring variable), and ones based on an understanding of some degradation process at play in the system. We will describe the first class in this section on predictive maintenance and the second class as condition-based maintenance in Section 11.4.4.2.

We have previously conceptualized the times at which failures of a maintainable system occur as a point process (Section 4.3.3). If it is possible to characterize the operating times in this point process thoroughly, what we know about the distributions of the operating times in the process may be used to construct a preventive maintenance schedule. For example, suppose n − 1 failures have occurred in the system so far. At the end of the current outage, the system is repaired and returned to service, and the next operating time U_n begins. If we know the distribution of U_n, we may choose a time at which we are, say, 90% certain that the next failure will occur beyond this time (this would be the 10th percentile of the distribution of U_n). Of course, this choice is not going to be arbitrary (although even an arbitrary choice has a chance of improving on the fixed schedule scheme) but will be determined through an optimization balancing the cost of carrying out the preventive maintenance too early against the costs of a failure. Some examples of optimization of predictive maintenance schemes can be found in Refs. 6, 14.

11.4.4.2 Condition-based maintenance

As an alternative to acquiring stochastic characterization of the system’s operating times, some physical characteristic(s) of the system may be measured and tracked over time⁷ in an attempt to predict when a failure may be imminent. The measurement may be passive (i.e., no special stimulus is applied to the system but rather some existing operating characteristic or sensor reading is followed) or active (i.e., some stimulus is applied to the system, and a response is measured).

In the first case, the measured characteristic is usually conceptualized as a stochastic process {X(t) : t ≥ 0} (X(t) is the value of the measurement at time t), and the connection with maintenance is that the system fails at the first time τ that this process crosses some stated threshold x₀. That is,

if we think of the process X(t) as nondecreasing. For instance, X(t) may represent the percentage of oxidation in a steel structure; when that percentage reaches a predetermined threshold x₀, some remedial action is taken to forestall collapse of the structure. X(t) may represent the level of vibration measured in some rotating machinery; when the measured vibration is too great, preventive action is taken to forestall a failure that may take place if bearing wear (or some other failure mechanism) were to be allowed to increase unchecked. A schedule of inspections is needed so that maintenance personnel will know when the system should be monitored. A static schedule calls for measurements at fixed, predetermined times. If the system is continuously monitored, this procedure is called condition monitoring [8] . A dynamic schedule may be developed based on an understanding of how rapidly X(t) changes, that is, slowly changing phenomena need not be inspected as often. A dynamic schedule may be created by estimating X′(t) at the same time X(t) is measured so that the time of the next inspection will be longer if X′(t) is small and shorter if X′(t) is large.

The measurements (longitudinal data) obtained from these inspections are values of X(t) at the inspection times t₁, t₂, . . . . Statistical treatment of these data in an engineering context was introduced by Carey and Tortorella [4] and developed to a high standard by Lu and Meeker [13] and others. Degradation analysis is now a standard part of statistical analysis of reliability data [15] and is widely used in many condition-based maintenance analyses [2, 12, 13, 14, 25].

In the second case, inspection entails measuring the response of the system to a defined stimulus. For example, inverse acoustic scattering may be used to detect cracks in structural materials [1], enabling early detection of deterioration that, if left unchecked, could lead to catastrophic failure [10] . In other respects, this procedure is like passive condition-based maintenance with the added feature that periodic inspection of elements not ordinarily visible may be accomplished.