Specialized features
This chapter describes the configuration of all specialized features that are available for IBM z14 ZR1. It includes the following topics:
10.1 Crypto Express6S
In this section, we describe the configuration of the Crypto Express6S feature on an IBM z14 ZR1. We also cover cryptographic domains, configuration rules, and what to consider when you are planning for nondisruptive installation of cryptographic features.
10.1.1 Crypto Express6S overview
The following generations of cryptographic coprocessors1 are supported in z14 ZR1:
•Crypto Express5S, Feature Code #0890, carry forward only (MES from z13s)
•Crypto Express6S, Feature Code #0893
This section describes the Crypto Express6S feature, with the new z14 ZR1. For more information about Crypto Express5S feature configuration (not changed from previous server generation [z13]), see IBM z13 Configuration Setup, SG24-8260.
Each of cryptographic coprocessor of z14 ZR1 includes 40 physical sets of registers, which correspond to the maximum number of LPARs running on a z14 ZR1. Each of these 40 sets belongs to the following domains:
•A cryptographic domain index, in the range of 0 - 39, is allocated to a logical partition by the definition of the partition in its image profile. The same domain must also be allocated to the ICSF instance running in the logical partition that uses the Options data set.
•Each ICSF instance accesses only the Master Keys that are corresponding to the domain number that is specified in the logical partition image profile at the Support Element and in its Options data set. Each ICSF instance sees a logical cryptographic coprocessor that consists of the physical cryptographic engine and the unique set of registers (the domain) that are allocated to this logical partition.
The installation of the CP Assist for Cryptographic Functions (CPACF) DES/TDES enablement (FC 3863) is one of the prerequisites for the use of the special hardware cryptographic feature in z14 ZR1. This feature enables the following functions:
•For data privacy and confidentially: Data Encryption Standard (DES) includes single-length key DES, double-length key DES, and triple-length key DES (also known as TDES). It also includes Advanced Encryption Standard (AES) for 128-bit, 192-bit, and 256-bit keys.
•For data integrity:
– Secure Hash Algorithm-1 (SHA-1) for 160-bit support
– SHA-2 for 224-bit, 256-bit, 384-bit, and 512-bit support
– SHA-3 for 224-bit, 256-bit, 384-bit, and 512-bit support
– SHAKE for 128-bit and 256-bit support
SHA-1, SHA-2, and SHA-3 are included enabled on all IBM z14 ZR1 and do not require the no-charge enablement FC 3863.
•For Key Generation: Pseudo-Random Number Generation (PRNG), Deterministic Random Number Generation (DRNG), and True Random number generation (TRNG).
•For message authentication code (MAC): Single-key MAC and double-key MAC.
The total number of Crypto Express6S or Crypto Express5S features cannot exceed 16 per z14 ZR1. The initial order for Crypto Express6S is two features (two PCIe adapters for Crypto Express6S). After the initial order, the minimum order is one feature.
Each Crypto Express6S feature contains one PCIe adapter. The adapter can be in the following configurations:
•Common Cryptographic Architecture (CCA) Coprocessor (CEX6C)
•Public Key Cryptography Standards (PKCS) #11 (EP11) Coprocessor (CEX6P)
•Accelerator (CEX6A)
During the feature installation, the PCIe adapter is configured by default as the CCA coprocessor.
The configuration of the Crypto Express6S adapter as EP11 coprocessor requires a Trusted Key Entry (TKE) workstation Hardware 9.0 (FC 0085 for the rack-mounted workstation, FC 0086 for the tower workstation) with TKE 9.0 Licensed Internal Code (FC 0879).
The Crypto Express6S feature does not use CHPIDs from the channel subsystem pool. However, the Crypto Express6S feature requires one slot in a PCIe I/O drawer, and one physical channel ID (PCHID) for each PCIe cryptographic adapter.
The cryptographic feature codes for z14 ZR1 are listed in Table 10-1.
Table 10-1 Cryptographic feature codes
|
Feature code
|
Description
|
|
3863
|
CP Assist for Cryptographic Functions (CPACF) enablement: This feature is a prerequisite to use CPACF (except for SHA-1, SHA-2, and SHA-3) and cryptographic coprocessor hardware.
|
|
0893
|
Crypto Express6S card: A maximum of 16 features can be ordered (minimum of two adapters). This feature is optional; each feature of which contains one PCI Express cryptographic adapter (adjunct processor). This feature is supported in z14 and z14 ZR1 only.
|
|
0890
|
Crypto Express5S card: This feature cannot be ordered for a new build z14 ZR1, but only on a carry forward MES from z13s. The maximum supported number of Crypto Express5S and Crypto Express6S is 16 in total. This feature is optional; each feature of which contains one PCI Express cryptographic adapter (adjunct processor). This feature is supported in z14, z14 ZR1, z13, and z13s servers only.
|
|
0086
|
Trusted Key Entry (TKE) tower workstation: A TKE provides basic key management (key identification, exchange, separation, update, and backup) and security administration. It is optional for running a Crypto Express6S card in CCA mode and required for running it in EP11 mode.
The TKE workstation includes one Ethernet port, and supports connectivity to an Ethernet local area network (LAN) operating at 10, 100, or 1000 Mbps. Up to 10 features per z14 ZR1 server can be ordered.
|
|
0085
|
Trusted Key Entry (TKE) rack-mounted workstation: The rack-mounted version of the TKE, which needs a customer-provided, standard 19-inch rack. It includes a 1U TKE unit and a 1U console tray (screen, keyboard, and pointing device). When smart card readers are used, another customer provided tray is needed. Up to 10 features per z14 ZR1 server can be ordered.
|
|
0879
|
TKE 9.0 Licensed Internal Code (LIC): Included with the TKE tower workstation FC 0086 and the TKE rack-mounted workstation FC 0085 for z14 ZR1. Earlier versions of TKE features (FCs 0842, 0847, 0097, and 0098) can also be upgraded to TKE 9.0 LIC.
|
|
0895
|
TKE Smart Card Reader: Access to information in the smart card is protected by a PIN. One feature code includes two smart card readers, two cables to connect to the TKE workstation, and 20 smart cards.
|
|
0892
|
TKE extra smart cards: When one feature code is ordered, 10 smart cards are included. The order increment is 1 - 99 (990 blank smart cards).
|
|
Note: You might need a TKE workstation that includes TKE Smart Card Reader while you run on CCA mode to satisfy certain industry security standard.
|
For more information about the Crypto Express6S feature and the corresponding crypto features, see the IBM z14 Technical Guide, SG24-8451.
10.1.2 Planning for Crypto Express6S configuration
|
Note: Support for Crypto Express6S coprocessors that are available on z14 ZR1 processors is included for z/OS V2.3, z/OS V2.2, and z/OS V2.1 in the Cryptographic Support for z/OS V2R1 – z/OS V2R3 (HCR77C1) web deliverable. It can be downloaded from the z/OS downloads website.
ICSF enhancements in z/OS V2.3 for the Crypto Express6S updates include support to use the new algorithm support and to extend existing support for asymmetric algorithms. This support also requires firmware or Microcode Change Level (MCL) updates to the TKE and z14 ZR1 processors, which are considered as co-requisites.
For more information about the latest MCL bundle requirements, see the Driver-27 Exception Letter that is available on IBM Resource Link (IBM ID authentication required).
|
The z14 ZR1 always operates in LPAR mode. The concept of dedicated coprocessor does not apply to the PCIe adapter. A PCIe adapter, whether configured as coprocessor or accelerator, is made available to logical partitions as directed by the domain assignment and the candidate list. This process occurs regardless of the shared or dedicated status that is given to the central processors in the partition.
The z14 ZR1 allows for up to 40 logical partitions to be active concurrently.
Each PCIe adapter on a Crypto Express6S feature supports 40 domains, whether it is configured as a Crypto Express6S coprocessor or a Crypto Express6S accelerator.
For availability reasons, the minimum configuration consists of two Crypto Express6S features so that every potential logical partition can have access to two cryptographic adapters.
More Crypto Express6S features might be needed to satisfy application performance and availability requirements. Consider the following points:
•For availability, spread assignment of multiple PCIe adapters of the same type (accelerator or coprocessor) to one logical partition across features in multiple I/O domains.
•The use of retained private keys on a PCIe adapter that is configured as a Crypto Express6S coprocessor creates an application single point of failure. This point of failure exists because RSA-retained private keys are not copied or backed up.
•An intrusion latch exists within the PCIe adapter logic that is set when the feature is removed from the system. If the feature is reinstalled and power is applied, the coprocessor keys and secrets are zeroed and the intrusion latch is reset.
If a TKE workstation is available, the PCIe adapter can first be disabled from the TKE workstation before you remove the feature from the system. In that case, when the feature is reinstalled, the coprocessor keys and secrets are not zeroed. The intrusion latch is reset, and the coprocessor remains in the disabled state. The PCIe adapter then can be enabled from the TKE and normal operations can be resumed.
Plan the definition of domain indexes and cryptographic coprocessor numbers in the Candidate list for each logical partition to prepare the cryptographic configuration. You can also define or change that cryptographic definition dynamically to an active logical partition with a running system. For more information, see “Changing LPAR Cryptographic Controls function” on page 225.
•Crypto Express6S features can be installed concurrently when all physical requirements are fulfilled. Dynamically enabling a new PCIe adapter to a partition requires the following configurations:
– At least one usage domain index is defined to the logical partition
– The cryptographic coprocessor numbers be defined in the partition Candidate list
•The same usage domain index can be defined more than once across multiple logical partitions. However, the cryptographic coprocessor number that is coupled with the specified usage domain index must be unique across all active logical partitions.
The same cryptographic coprocessor number and usage domain index combination can be defined for more than one logical partition. This feature can be used, for example, to define a configuration for backup situations. In this case, only one of the logical partitions can be active at any one time.
•Newly installed Crypto Express6S features are assigned coprocessor numbers sequentially during the power-on-reset that follows the installation.
However, when a Crypto Express6S feature is installed concurrently by using the Nondisruptive Hardware Change task, the installation might select an out-of-sequence coprocessor number from the unused range. In this case, communicate the cryptographic coprocessor numbers that you want to the IBM installation team.
When the task is used to concurrently remove a PCI cryptographic feature, the coprocessor number is automatically freed.
Table 10-2 is a simplified configuration map. Each row identifies a PCIe adapter, and each column identifies a domain index number. Each cell entry indicates the logical partition to be assigned the cryptographic coprocessor number that is coupled with the usage domain index.
Table 10-2 Planning for logical partitions, domains, and PCIe adapter numbers
|
|
Domain index 0
|
Domain index 1
|
Domain index 2
|
.../...
|
Domain index 39
|
|
PCIe adapter 0
|
LP00
LP02
|
LP04
|
LP05
|
|
|
|
PCIe adapter 1
|
LP01
LP02
|
|
|
|
|
|
PCIe adapter 2
|
|
|
|
|
|
|
…
|
|
|
|
|
|
|
…
|
|
|
|
|
|
|
PCIe adapter 13
|
|
|
|
|
|
|
PCIe adapter 14
|
|
|
|
|
|
|
PCIe adapter 15
|
|
|
|
|
|
Table 10-2 on page 219 lists the following characteristics:
•Logical partitions LP00 and LP01 use domain 0 (zero), but are assigned different PCIe adapters. No conflict exists and they can be concurrently active.
•Logical partition LP02 uses domain 0 (zero) on the set of cryptographic adapters that are already defined to LP00 and LP01. Therefore, LP02 cannot be active concurrently with either LP00 or LP01. However, the definition might be valid for backup situations.
•Logical partitions LP04 and LP05 use different domain numbers for PCIe cryptographic adapter 0 (zero); therefore, no conflict exists. The combination of domain number and cryptographic coprocessor number is unique across partitions.
|
Important: Any combination of PCIe adapter and domain index should contain only one active logical partition. The combination of cryptographic coprocessor number and usage domain index must be unique across all active logical partitions.
|
For more information about the Crypto Express6S feature for IBM Z, see IBM z14 Technical Guide, SG24-8451.
10.1.3 Configuring Crypto Express6S
This section describes the steps for configuring Crypto Express6S for the IBM z14 ZR1 server.
The z14 ZR1 operates in LPAR mode only. For each logical partition that requires access to a PCIe adapter (configured as an accelerator or coprocessor), the required information must be defined in the partition Image profile. This technique ensures the correct use of the cryptographic features when the associated partition is activated.
Concurrent changes to the Crypto Express6S features and controls when the partition is already activated are provided by special functions at the Support Element (SE).
Checking whether CPACF DES/TDES enablement feature is installed
The z14 ZR1 FC 3863 enables the DES and TDES algorithms on the CPACF. It is one of the prerequisites for using the Crypto Express6S feature. You must verify whether the CPACF feature is properly installed on the processor before you configure cryptographic functions. This information is displayed in the SE, and can be verified by completing the following steps:
1. Log on to the SE directly, or click Single Object Operations from the HMC.
2. Open the System details menu of the CPC at the SE workplace. The system details window opens (see Figure 10-1 on page 221).
Figure 10-1 System details: CPACF installed
3. Click the Instance Information tab and verify that the CPACF DES/TDES enablement feature code 3863 is installed:
– If the window shows the message CP Assist for Crypto Functions: Installed, the CPACF enablement FC 3863 is enabled.
– If the window shows the message CP Assist for Crypto Functions: Not installed, FC 3863 is not installed. You can still customize the partition image profiles, but the cryptographic functions do not operate.
Logical partition cryptographic definition
The next step is to define the following cryptographic resources in the image profile for each partition:
•Usage domain index
•Control domain index
•PCI Cryptographic Coprocessor Candidate List
•PCI Cryptographic Coprocessor Online List
These resources are defined by using the Customize/Delete Activation Profile task, which is in the Operational Customization Group, either from the HMC or from the SE. Modify the cryptographic initial definition from the Crypto option in the image profile, as shown in Figure 10-2 on page 222. After this definition is modified, any change to the image profile requires a DEACTIVATE and ACTIVATE of the logical partition for the change to take effect. Therefore, this kind of cryptographic definition is disruptive to a running system.
|
Tip: Operational changes can be made by using the Change LPAR Cryptographic Controls task from the SE, which reflects the cryptographic definitions in the image profile for the partition. With this function, you can dynamically add and remove the cryptographic feature without stopping a running operating system. For more information about using this function, see “Changing LPAR Cryptographic Controls function” on page 225.
|
Figure 10-2 Customize Image Profiles: Crypto
The cryptographic resource definitions include the following meanings:
•Control Domain
Identifies the cryptographic coprocessor domains that can be administered from this logical partition if it is being set-up as the TCP/IP host for the TKE.
If you are setting up the host TCP/IP in this logical partition to communicate with the TKE, the partition is used as a path to other domains’ Master Keys. Indicate all the control domains that you want to access (including this partition’s own control domain) from this partition.
•Control and Usage Domain
Identifies the cryptographic coprocessor domains that are assigned to the partition for all cryptographic coprocessors that are configured on the partition. The usage domains cannot be removed if they are online.
The numbers that are selected must match the domain numbers that are entered in the Options data set when you start this partition instance of ICSF.
The same usage domain index can be used by multiple partitions regardless to which CSS they are defined. However, the combination of PCIe adapter number and usage domain index number must be unique across all active partitions.
•Cryptographic Candidate list
Identifies the cryptographic coprocessor numbers that are eligible to be accessed by this logical partition. From the list, select the coprocessor numbers (in the range 0 - 15) that identify the PCIe adapters to be accessed by this partition.
No error condition is reported when a cryptographic coprocessor number, which is selected in the partition candidate list, is available to the partition when the partition is activated because it is configured off or not installed. The cryptographic coprocessor number is ignored and the activation process continues.
If the cryptographic coprocessor number and usage domain index combination for the coprocessor that is selected is in use by another active logical partition, the activation of the logical partition fails (see Figure 10-3).
Figure 10-3 Activation of LPAR failed: ACTZ01DD
In this conflicting case, you must review the cryptographic information for all active logical partitions from the Summary tab of the View LPAR Cryptographic Controls task (see Figure 10-5 on page 225). Resolve the error based on the collected data by assigning a unique combination of PCIe adapter number and usage domain index number.
•Cryptographic Online list
Identifies the cryptographic coprocessor numbers that are automatically brought online during logical partition activation. The numbers that are selected in the online list must also be part of the candidate list.
After the next partition activation, installed PCI Cryptographic Coprocessors that are in the partition’s PCI Cryptographic Coprocessor Candidate list but not on the PCI Cryptographic Coprocessor Online list are in a configured off state (Standby). They can be configured online later to the partition by selecting Configure On/Off from the SE. For more information, see “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
When the partition is activated, no error condition is reported if a cryptographic coprocessor number that is selected in the partition’s online list is not installed. The cryptographic coprocessor is ignored and the activation process continues.
When a cryptographic coprocessor number that was selected in the partition’s online list was configured to an off state to the partition, it is automatically configured back to an on state when the partition is activated. The cryptographic online list is always selected from the image profile for each logical partition.
Cryptographic configuration using the Support Element
You can complete the following tasks from the SE:
•Display the PCI Cryptographic Configuration.
•Display the LPAR cryptographic controls (domain index and candidate or online lists for currently activate partitions).
•Reconfigure the coprocessor from or to the accelerator.
•Configure a cryptographic coprocessor and accelerator on or off to a logical partition.
•Change LPAR cryptographic controls to a logical partition.
These tasks require you to work from the SE. To get to the appropriate SE task, log on to the SE directly, or click Single Object Operations from the HMC.
Cryptographic management
After you select the CPCs, click Cryptographic Management in the Configuration section.
Figure 10-4 shows the Cryptographic Management window. Use this window to obtain the installed cryptographic configuration (the association of the cryptographic number and the card serial number). The following options are available:
•View installed cryptographic features, with status and assigned PCHID and coprocessor numbers. Each PCIe adapter is assigned to a coprocessor number, in the range 0 - 15, as part of the configuration process. The assignment is made when the feature is installed.
•View coprocessor numbers that are still assigned to removed cryptographic features.
•Start the release of coprocessor numbers. Remove the relationship only when a Crypto Express6S feature is permanently removed from the CPC.
The release option removes the relationship between a PCI cryptographic feature serial number and the assigned coprocessor numbers. Removing the relationship frees the coprocessor numbers, which makes them available to be assigned to a new feature serial number.
|
Important: The coprocessor numbers are assigned to the feature serial number, not to the installed location. If a feature is removed from one location to be reinstalled in another, the coprocessor number assignment remains.
|
Figure 10-4 SE Cryptographic Management
Viewing LPAR Cryptographic Controls task
You can view active partition cryptographic definitions from the SE.
Select the CPCs and click View LPAR Cryptographic Controls in the Operational Customization window.
The resulting window displays the definition of Usage and Control domain indexes, and PCI Cryptographic candidate and online lists. The information is provided for active logical partitions only.
|
Tip: You can review the PCI Cryptographic candidate lists and usage domain indexes that are assigned for all active logical partition from the Summary tab (see Figure 10-5). The usage domain index, in combination with the cryptographic number that is selected in the candidate list, must be unique across all partitions that are defined to the CPC. Therefore, this new tab is useful when you define or change the usage domain index for a logical partition.
|
Figure 10-5 View LPAR Cryptographic Controls
This window is for informational purposes only. You can see the definitions, but you cannot change them by using this window. Modifying the cryptographic coprocessor on/off status requires the use of the Configure On/Off task, which is described in “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
Changing LPAR Cryptographic Controls function
For each logical partition, you can define the following attributes:
•Usage domain index
•Control domain index
•Cryptographic Coprocessor Candidate list
•Cryptographic Coprocessor Online list
You can complete the following tasks by using the Change LPAR Cryptographic Controls function, which is included in the SE for the z14 ZR1:
•Add a cryptographic coprocessor to a logical partition for the first time.
•Add a cryptographic coprocessor to a logical partition that uses a cryptographic coprocessor.
•Remove a cryptographic coprocessor from a logical partition.
•Zeroize or clear the cryptographic secure keys for a usage domain.
Dynamic assignment of the cryptographic definition to the partition
All the cryptographic functions that are defined in the Image Profile can be dynamically changed by using the Change LPAR Cryptographic Controls window at the SE. For more information about defining functions in the Image Profile, see “Logical partition cryptographic definition” on page 221.
Select the Control and Usage for each domain and the cryptographic Candidate and Online for each Crypto (see Figure 10-6).
Figure 10-6 Change LPAR Cryptographic Controls: Change Running System
After selecting the appropriate options, you can complete the following tasks:
•Save these settings to the Image Profile without changing the running system.
•Change the running system without saving the definition to the Image Profile, which means your changes are lost after a reactivation of the partition.
|
Remember: Changes to the Cryptographic Coprocessor Online List are ignored when this option is selected.
|
•Save the definitions to the Image Profile and activate the changes immediately to the partition.
When you add or change the control or usage domain index and cryptographic coprocessor number dynamically for a running system, a confirmation message is displayed. After processing, a status window opens that indicates the result of a dynamic addition or change of a cryptographic definition to an LPAR (see Figure 10-7).
Figure 10-7 SE: Change LPAR Cryptographic Controls
Dynamic removal of the cryptographic definition
You can remove the cryptographic definition from a logical partition dynamically by using the Change LPAR Cryptographic Controls task. This section addresses the related issues and describes the procedure.
Complete the following steps:
1. Before you change the cryptographic settings by using the Change LPAR Cryptographic Controls task, verify that the cryptographic lists you want to remove from a logical partition are offline (Standby). For more information about setting the cryptographic channel status, see “Configuring a Crypto Express6S online or offline on a logical partition” on page 235. If you try to remove the lists dynamically while they are online, the change fails and you receive the message that is shown in Figure 10-8.
Figure 10-8 SE: Change LPAR Cryptographic Controls: ACT33679
In addition to adding or changing cryptographic settings for a logical partition, you can remove the Control and Usage domains or Cryptographic Candidate lists for a logical partition from the Change LPAR Cryptographic Controls window (see Figure 10-6 on page 226).
After clearing the definitions for a logical partition, remove a definition dynamically by clicking Change Running System. To save the new configuration to the Image Profile without changing the running system, select Save to Profiles. With Save and Change, the removal becomes concurrently active, and the removed cryptographic coprocessor also cannot be used for the next image activation.
2. When you remove the only definition of the cryptographic lists, the Usage Domain Zeroize window opens (see Figure 10-9).
|
Consideration: Because you cannot see all cryptographic information, including the usage domains for other logical partitions, you might need to check the information in View LPAR Cryptographic Controls window before you continue. For more information about zeroize, see “Reconfiguring the PCIe Adapter type” on page 228.
|
Figure 10-9 SE: Change LPAR Cryptographic Controls Zeroize
3. In the confirmation window (see Figure 10-10), click OK to dynamically change the cryptographic settings. After processing, a status window indicates the result of the dynamic change of cryptographic definition to an LPAR.
Figure 10-10 SE: Change LPAR Cryptographic Controls: ACT33680
Reconfiguring the PCIe Adapter type
Each PCIe Crypto Express6S feature can be configured as a coprocessor or an accelerator. Each Crypto Express6S feature can be set in the following configurations:
•Common Cryptographic Architecture (CCA) Coprocessor (CEX6C)
•Public Key Cryptography Standards #11 (PKCS) (EP11) Coprocessor (CEX6P)
•Accelerator (CEX6A)
Whether it is configured as a coprocessor or an accelerator, each PCIe Cryptographic adapter can be shared among 40 logical partitions.
Configuring a CCA coprocessor as an accelerator
During the installation of a Crypto Express6S feature, the PCIe Cryptographic adapter is configured by default as a CCA coprocessor. The reconfiguration is fully supported in Licensed Internal Code.
When a PCIe adapter is configured as a CCA coprocessor, it can still run accelerator functions, although much more slowly than when configured as accelerator. When it is configured as an accelerator, it cannot run coprocessor functions.
When a PCIe adapter is configured as an EP11 coprocessor, a TKE workstation is required for the management of the Crypto Express6S. For more information about configuring EP11 coprocessor, see “Configuring a CCA coprocessor as an EP11 coprocessor” on page 232.
To reconfigure the PCIe Adapter from coprocessor to accelerator, complete the following steps:
1. Select the CPC that has cryptographic coprocessor adapters that you want to reconfigure, and then, click the Cryptographic Configuration task in the Configuration Group.
2. The reconfiguration is enabled only for PCIe adapters that are Off. Therefore, be sure that the PCIe Cryptographic adapter status for that cryptographic coprocessor channel is unconfigured. If necessary, set the PCIe Cryptographic adapter to Off for all partitions that have it in their candidate list. To set the PCIe Cryptographic adapter to Off, use the procedure that is described in “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
3. Select the number of the cryptographic coprocessor channel (see Figure 10-11) and click Crypto Type Configuration.
Figure 10-11 Cryptographic Configuration task (unconfigured)
4. Change the configuration for the cryptographic coprocessor adapter. The selected cryptographic coprocessor channel is configured as a coprocessor (see Figure 10-12). Select Accelerator.
Figure 10-12 Crypto Type Configuration (CCA coprocessor to Accelerator)
By selecting Accelerator, you can zeroize the selected coprocessor by also selecting Zeroize the Coprocessor on the Crypto Type Configuration window. However, click to the Zeroize the Coprocessor option and then, click OK.
|
Important: Zeroizing one or all cryptographic coprocessors clears their configuration data and all cryptographic keys. Zeroizing also erases configuration data from the SE hard disk drive (for example, UDX files). Zeroize cryptographic coprocessors manually only when necessary (typically, when the cryptographic coprocessor configuration data must be erased completely). In normal cases, be sure to clear the check box for each cryptographic channel.
|
Figure 10-13 Crypto Type Configuration Confirmation for accelerator
6. Verify that your request completed successfully. Click OK.
7. You are returned to the Crypto Type Configuration window. Click Cancel. You are returned to the Cryptographic Configuration window. Confirm that the target cryptographic channel changed to the cryptographic accelerator type. The Crypto Serial Number, Operating mode, and TKE Commands should be Not available until the cryptography is set to Online again, as described in “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
After you perform this task and return to the Cryptographic Configuration window, the information that is shown in Figure 10-14 is displayed.
|
Note: UDX support is not available for Crypto Express6S that is defined as an EP11 coprocessor and accelerator.
|
Figure 10-14 Cryptographic Configuration (Accelerator online)
Figure 10-15 Cryptographic Details (Accelerator)
The Cryptographic Type is now a Crypto Express6S Accelerator. The adapter was not zeroized during the type-changing procedure.
The procedure for changing the type of the cryptographic configuration from an accelerator to a coprocessor is now complete. To change the accelerator back to a coprocessor, the same procedure can be used, but select Coprocessor instead of Accelerator, as shown in Figure 10-12 on page 230.
The result of this change is shown in Figure 10-16.
Figure 10-16 SE: Cryptographic Details (CCA Coprocessor)
Configuring a CCA coprocessor as an EP11 coprocessor
To configure a CCA coprocessor as an EP11 coprocessor, complete the following steps:
1. Select the CPC that includes cryptographic coprocessor adapters that you want to reconfigure and click Cryptographic Configuration in the CPC Configuration Group.
2. Because the reconfiguration is enabled only for PCIe adapters that are set to Off, be sure the PCIe Cryptographic adapter status for that cryptographic coprocessor channel is unconfigured (see Figure 10-12 on page 230).
If necessary, set the PCIe Cryptographic adapter to Off for all partitions that are included in their candidate list. For more information about setting the PCIe Cryptographic adapter to Off, see “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
3. Select the number of the cryptographic coprocessor channel and click Crypto Type Configuration.
4. Change the configuration for the cryptographic coprocessor adapter. Select EP11 Coprocessor (see Figure 10-17), which by default, automatically selects the Zeroize the coprocessor option. Click OK.
Figure 10-17 SE Crypto Type Configuration (CCA Coprocessor to EP11 Coprocessor)
Figure 10-18 Crypto Type Configuration Confirmation for EP11 Coprocessor
6. Check that your request completed successfully. Click OK to return to the Crypto Type Configuration window.
7. Click Cancel in the Crypto Type Configuration window to return to the Cryptographic Configuration window. You can confirm that the target cryptographic channel changed to the EP11 Coprocessor type in the Cryptographic Configuration task window. The Crypto Serial Number, Operating mode, and TKE Commands should be Not available until the cryptography is set to Online again, as described in “Configuring a Crypto Express6S online or offline on a logical partition” on page 235.
After you complete this task and return to the Cryptographic Configuration window, the information that is shown in Figure 10-19 is displayed.
Figure 10-19 SE: Cryptographic Configuration (EP11 Coprocessor online)
Figure 10-20 SE Cryptographic details (EP11 Coprocessor)
The Cryptographic Type is now a Crypto Express6S EP11 Coprocessor.
This process completes changing the type of the cryptographic configuration from a CCA Coprocessor to an EP11 coprocessor. To change the configuration back to CCA Coprocessor, the same procedure can be used, but select CCA Coprocessor instead of EP11 Coprocessor.
You can also switch the configuration mode from Accelerator to EP11 Coprocessor and from EP11 to Accelerator by using the same process, but selecting Accelerator or EP11 Coprocessor as required.
|
Requirement: To manage a Crypto Express6S feature that is configured as an EP11 coprocessor, the TKE workstation is required.
|
Configuring a Crypto Express6S online or offline on a logical partition
For some changes to the cryptographic settings to the logical partition, you must configure the Crypto Express6S online or offline. This dynamic operation is not needed if you can reactivate (DEACTIVATE and ACTIVATE) the image for the logical partitions whose cryptographic online lists were updated.
Setting a Crypto Express6S to an online state
To set a Crypto Express6S online, complete the following steps:
1. From the SE, select the System Management function.
2. Select the server, click Partitions, and then, select the logical partition.
3. Click the Cryptos selection for the target logical partition.
4. In the contents of Cryptos page, select the Crypto IDs to be changed. Figure 10-21 shows that on server MUSCA to Logical Partition MUSCA11, two cryptographic coprocessors are defined: One CCA coprocessor (ID 00, physical Channel ID 0104), and one Accelerator (ID01, physical Channel ID 0164). The CCA coprocessor is online, and the Accelerator is offline. We now want to also set the Accelerator online.
Figure 10-21 System Management: LPAR Crypto Selection, Standby
5. Select the cryptographic coprocessor and click Tasks → Crypto Service Operations → Configure On/Off task (see Figure 10-22). This task controls the online or offline (standby) state of a cryptographic processor for logical partitions that are defined in the cryptographic processor’s candidate list.
Figure 10-22 Crypto Service Operations: LPAR Crypto Selection, Configure Online
6. Select the cryptographic coprocessor channel number that you want, and then, click Select Action → Toggle to switch from Standby to Online (see Figure 10-23). If you want multiple cryptographic channels at the same time, select Toggle All On.
Figure 10-23 Config On/Off (Standby)
7. After confirming that your requested cryptographic coprocessor channel is set to the wanted state of Online, click OK (see Figure 10-24).
Figure 10-24 Configure On/Off (Standby to Online)
Figure 10-25 Configure On/Off (Standby to Online) completed
9. After you verify that the current state of the channels changed to Online, click Cancel to return.
You can view the contents of the Cryptos window of the logical partition to confirm that the cryptographic channels are now in the Operating status (see Figure 10-26).
Figure 10-26 System Management: LPAR Crypto Selection, Online
Changing a cryptographic channel to standby (offline) status
To change the cryptographic channel status, complete the following steps:
1. Select the logical partition whose Crypto IDs you want to change to Standby. For example, select the Accelerator (01), that is in an online state. Click Tasks → Crypto Service Operations → Configure On/Off task (see Figure 10-27).
Figure 10-27 System Management: LPAR Crypto Selection, Configure Offline
2. Select the cryptographic coprocessor channel number that you want, and click Select Action → Toggle All Standby to switch from Online to Standby (see Figure 10-28).
Figure 10-28 Config On/Off (Online)
3. After you confirm that the state for your requested cryptographic channel is Standby, click OK (see Figure 10-29).
Figure 10-29 Configure On/Off (Online to Standby)
4. Because taking a cryptographic coprocessor offline can be disruptive to your application, a confirmation is required. The task must be confirmed by keying in the user password (Figure 10-30).
Figure 10-30 Configure On/Off (Online to Standby): Confirmation
Figure 10-31 Config On/Off (Online to Standby) completed
10.1.4 Handling cryptographic coprocessors by using ICSF
Integrated cryptographic service facility (ICSF) provides an Interactive System Productivity Facility (ISPF) Coprocessor Management panel in which you can display or change the status (Active or Deactivate) of cryptographic coprocessors. This action affects only the coprocessor status of ICSF, and has no effect on the Online/Standby hardware status that is displayed on the z14 ZR1 SE.
It is not the purpose of this section to show how to create, load, and manage keys in the cryptographic adapters. For that information, see the ICSF literature. This section shows only how to activate and deactivate a cryptographic coprocessor, and how to display the hardware status.
From the ICSF main panel (see Figure 10-32), select option 1 to open the ICSF Coprocessor Management panel.
|
HCR77C1 -------------- Integrated Cryptographic Service Facility -------------
OPTION ===>
System Name: SC03 Crypto Domain: 39
Enter the number of the desired option.
1 COPROCESSOR MGMT - Management of Cryptographic Coprocessors
2 KDS MANAGEMENT - Master key set or change, KDS Processing
3 OPSTAT - Installation options
4 ADMINCNTL - Administrative Control Functions
5 UTILITY - ICSF Utilities
6 PPINIT - Pass Phrase Master Key/KDS Initialization
7 TKE - TKE PKA Direct Key Load
8 KGUP - Key Generator Utility processes
9 UDX MGMT - Management of User Defined Extensions
Licensed Materials - Property of IBM
5650-ZOS Copyright IBM Corp. 1989, 2017.
US Government Users Restricted Rights - Use, duplication or
disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Press ENTER to go to the selected option.
Press END to exit to the previous menu.
|
Figure 10-32 Integrated Cryptographic Support Facility main panel
Cryptographic coprocessors that are configured on the partition are listed in the ICSF Coprocessor Management panel (see Figure 10-33).
|
------------------------- ICSF Coprocessor Management -------- Row 1 to 2 of 2
COMMAND ===> SCROLL ===> PAGE
Select the cryptographic features to be processed and press ENTER.
Action characters are: A, D, E, K, R, S and V. See the help panel for details.
CRYPTO SERIAL
FEATURE NUMBER STATUS AES DES ECC RSA P11
------- -------- -------------------- --- --- --- --- ---
. 6P00 DV7CG305 Active I
. 6A01 N/A Active
******************************* Bottom of data ********************************
|
Figure 10-33 ICSF Coprocessor Management
When a coprocessor is configured offline to the logical partition from the SE (standby status), it is shown as Offline in the ICSF Coprocessor Management panel (see Figure 10-34).
|
------------------------- ICSF Coprocessor Management -------- Row 1 to 2 of 2
COMMAND ===> SCROLL ===> PAGE
Select the cryptographic features to be processed and press ENTER.
Action characters are: A, D, E, K, R, S and V. See the help panel for details.
CRYPTO SERIAL
FEATURE NUMBER STATUS AES DES ECC RSA P11
------- -------- -------------------- --- --- --- --- ---
. 6P00 DV7CG305 Active I
. 6A01 N/A Offline
******************************* Bottom of data ********************************
|
Figure 10-34 ICSF Coprocessor Management (Candidate only - Standby)
A cryptographic coprocessor becomes visible to ICSF Coprocessor Management when the coprocessor number is part of the partition candidate list and the coprocessor is first brought online to the partition in either of the following ways:
•At the time the partition is activated, if the coprocessor is installed and the coprocessor number is part of the partition Online list.
•When the coprocessor is first configured online to the partition by using the Config On/Off task from the SE Workplace.
In the list (see Figure 10-35), enter A or D to switch a coprocessor status to Active or Deactivated.
|
------------------------- ICSF Coprocessor Management -------- Row 1 to 2 of 2
COMMAND ===> SCROLL ===> PAGE
Select the cryptographic features to be processed and press ENTER.
Action characters are: A, D, E, K, R, S and V. See the help panel for details.
CRYPTO SERIAL
FEATURE NUMBER STATUS AES DES ECC RSA P11
------- -------- -------------------- --- --- --- --- ---
. 6P00 DV7CG305 Active I
d 6A01 N/A Active
******************************* Bottom of data ********************************
|
Figure 10-35 ICSF Coprocessor Management (Online)
When a coprocessor is deactivated through IC0SF (see Figure 10-36), it cannot be used by applications that run in that system image. The EP11 coprocessor configuration requires a TKE workstation.
Generally, deactivate an active coprocessor from the ICSF Coprocessor Management panel before it is configured off from the SE.
|
Note: If you do not deactivate the coprocessor first, some jobs might not be rerouted correctly.
|
|
------------------------- ICSF Coprocessor Management -------- Row 1 to 2 of 2
COMMAND ===> SCROLL ===> PAGE
Select the cryptographic features to be processed and press ENTER.
Action characters are: A, D, E, K, R, S and V. See the help panel for details.
CRYPTO SERIAL
FEATURE NUMBER STATUS AES DES ECC RSA P11
------- -------- -------------------- --- --- --- --- ---
. 6P00 DV7CG305 Active I
. 6A01 N/A Deactivated
******************************* Bottom of data ********************************
|
Figure 10-36 SF Coprocessor Management (Deactivated)
The Active/Deactivated status viewed from ICSF Coprocessor Management does not change the Online/Standby status that is set from the z14 ZR1 SE.
The ICSF Coprocessor Hardware Status panel is shown in Figure 10-37.
|
----------------- ICSF - PKCS #11 Coprocessor Hardware Status -----------------
COMMAND ===> SCROLL ===>
CRYPTO DOMAIN: 39
REGISTER STATUS COPROCESSOR 6P00
Crypto Serial Number : DV7CG305
Status : ACTIVE
Compliance Mode : FIPS: 2009
: BSI: NONE
P11 Master Key
New Master Key register : EMPTY
Verification pattern :
:
Current Master Key register : EMPTY
Verification pattern :
:
Press ENTER to refresh the hardware status display.
Press END to exit to the previous menu.
|
Figure 10-37 ICSF Coprocessor Hardware Status
Help information from ICSF Coprocessor Management (see Figure 10-38 and Figure 10-39 on page 243) describes valid actions and status information for each type of cryptographic coprocessor.
|
----------------------- Help for Coprocessor Management -----------------------
COMMAND ===>
Press enter to page through this help.
More: +
The Coprocessor Management panel displays the status of all cryptographic
coprocessors installed. Select the coprocessors to be processed.
Prefix Type of cryptographic coprocessor Valid action characters
------ --------------------------------- -----------------------
A PCI Crypto Accelerator a, d
X PCI X Crypto Coprocessor a, d, e, k, r, s, v
2C Crypto Express2 Coprocessor a, d, e, k, r, s, v
2A Crypto Express2 Accelerator a, d,
3C Crypto Express3 Coprocessor a, d, e, k, r, s, v
3A Crypto Express3 Accelerator a, d
4A Crypto Express4 Accelerator a, d
4C Crypto Express4 CCA coprocessor a, d, e, k, r, s, v
4P Crypto Express4 PKCS #11 coprocessor a, d, r, s
5A Crypto Express5 Accelerator a, d
5C Crypto Express5 CCA coprocessor a, d, e, k, r, s, v
5P Crypto Express5 PKCS #11 coprocessor a, d, r, s
6A Crypto Express6 Accelerator a, d
6C Crypto Express6 CCA coprocessor a, d, e, k, r, s, v
6P Crypto Express6 PKCS #11 coprocessor a, d, r, s
Action characters: (entered on the left of the coprocessor number)
'a' Makes available a coprocessor previously deactivated by a 'd'.
'd' Makes a coprocessor unavailable.
'e' Selects the coprocessor for master key entry.
'k' Selects the coprocessor for operational key load.
'r' Causes the coprocessor domain role to be displayed.
's' Causes complete hardware status to be displayed.
'v' Causes the coprocessor default role to be displayed with offsets.
The action character 'e' can not be combined with any other action characters.
The action character 'k' may be specified on only one coprocessor.
The action character 's' may not be specified for both CCA and
PKCS #11 coprocessors at the same time.
Status:
- Active: The feature is available for work.
- Offline: The feature is installed but not available to ICSF.
- Deactivated: The feature has been deactivated (see action
characters)
- Busy: The feature is temporarily busy.
- Hardware error: The feature has been stopped.
- Disabled by TKE: The feature has removed from service by a TKE
workstation.
- Master key incorrect: At least one master key is incorrect.
- Being reconfigured: An error has been detected and being checked by the
configuration task
- Initializing stage 1: The feature has been detected by the configuration
task. No status is available.
|
Figure 10-38 Help for Coprocessor Management (part 1 of 2)
|
- Initializing stage 2: The feature is being reset by the configuration
task. No status is available.
- Initializing stage 3: The feature is being readied by configuration
task. No status is available.
- Unknown response: The feature has returned a return/reason code
combination that ICSF does not recognize.
- Hung User on Feature: The feature is not responding. One or more users
hold the feature latch. If this problem persists
please take a dump and contact IBM service.
You will need to recycle ICSF to reclaim use of
the feature.
- Bad feature response: An unexpected response was received from a feature.
- Retry limit reached: Initialization of the feature failed.
- Unknown feature type: A feature has a type that is not recognized by ICSF.
- Unknown feature type: A feature has a type that is not recognized by ICSF.
- Repeat failures: A feature has experienced repeated failures
and recovered. The feature is made inactive and
will require manual intervention to cause ICSF
to attempt to use it again.
Cryptographic Coprocessor Master Key State:
A: Master key Verification Pattern matches the Key Store (CKDS, PKDS, or
TKDS) and the master key is available for use
C: Master key Verification Pattern matches the Key Store, but the master
key is not available for use
E: Master key Verification Pattern mismatch for Key Store or, for P11, no
TKDS was specified in the options data set
I: The Master key Verification Pattern in the Key Store is not set,
so the contents of the Master key are Ignored
U: Master key is not initialized
-: Not supported
: Not applicable
F3 = END HELP
|
Figure 10-39 Help for Coprocessor Management (part 2 of 2)
10.2 zEnterprise Data Compression feature
This section describes the configuration of the zEnterprise Data Compression (zEDC) feature on an IBM z14 ZR1.
10.2.1 zEDC overview
The zEDC Express is an optional feature that is available with the zEC12, zBC12, z13, z13s, z14, and z14 ZR1. It is designed to provide hardware-based acceleration for data compression and decompression.
The zEDC Express feature is a native PCIe I/O card that can be installed in the PCIe I/O drawer, up to four zEDC Express features per drawer domain. A total of 1 - 8 features can be installed on the system. One PCIe adapter/compression coprocessor is available per feature, and a zEDC Express feature can be shared by up to 15 LPARs.
For more information about the zEDC feature, see IBM z14 Technical Guide, SG24-8451, and Reduce Storage Occupancy and Increase Operations Efficiency with IBM zEnterprise Data Compression, SG24-8259.
10.2.2 Planning for zEDC configuration
For more information about planning, see 2.8.8, “Planning considerations for zEDC” on page 29.
10.2.3 Configuring zEDC
For more information about configuring a zEDC card by using HCD, see 15.2.4, “Defining a zEDC EXPRESS PCIe function” on page 379.
10.2.4 Handling zEDC
This section briefly describes how to manage the zEDC Express feature.
To verify that the IFAPRDxx member in your SYS1.PARMLIB concatenation is updated with the zEDC contents, use the D IQP command.
|
Note: Schedule an IPL for every LPAR that you want zEDC active on. For more information, see Reduce Storage Occupancy and Increase Operations Efficiency with IBM zEnterprise Data Compression, SG24-8259.
|
Example 10-1 shows the output of the DISPLAY IQP command that shows the status of the software feature.
Example 10-1 Display Feature Enablement: Enabled
DISPLAY IQP
IQP066I 15.09.38 DISPLAY IQP 961
zEDC Information
MAXSEGMENTS: 4 (64M)
Previous MAXSEGMENTS: N/A
Allocated segments: 1 (16M)
Used segments: 0 (0M)
DEFMINREQSIZE: 4K
INFMINREQSIZE: 16K
Feature Enablement: Enabled
Verify that the hardware features are configured for the specific LPAR by displaying the status of the PCIe functions, as shown in Example 10-2.
Example 10-2 Display PCIe status - Hardware Accelerator: STNBY
DISPLAY PCIE
IQP022I 15.31.00 DISPLAY PCIE 981
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000A2 RoCE Express2 CNFG 0100 0003 1
000000A3 RoCE Express2 CNFG 0100 0004 2
00000102 8GB zHyperLink CNFG 0108 0003 1
00000103 8GB zHyperLink CNFG 0108 0004 2
000000D1 Hardware Accelerator STNBY 0140 0002
000000C1 Hardware Accelerator STNBY 0120 0002
The STNBY status denotes that the device is present and in standby mode and ready to be configured online.
Bring the device online by using the CONFIG PFID(xx),ONLINE command, as shown in Example 10-3.
Example 10-3 Configure PFID online
CONFIG PFID(D1),ONLINE
IEE504I PFID(D1),ONLINE
IEE712I CONFIG PROCESSING COMPLETE
Verification of the PCIe status shows the new status of the PCIe functions, as shown in Example 10-4.
Example 10-4 Display PCIe status - Hardware Accelerator: ALLC
DISPLAY PCIE
IQP022I 15.45.52 DISPLAY PCIE 991
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000A2 RoCE Express2 CNFG 0100 0003 1
000000A3 RoCE Express2 CNFG 0100 0004 2
00000102 8GB zHyperLink CNFG 0108 0003 1
00000103 8GB zHyperLink CNFG 0108 0004 2
000000D1 Hardware Accelerator ALLC 0012 FPGHWAM 0140 0002
000000C1 Hardware Accelerator STNBY 0120 0002
Notice that address space FPGHWAM (see Example 10-5) was started automatically by the system. Also, address space PCIe is started. Both address spaces are mandatory to handle PCIe functions.
Example 10-5 Display PCIe status extended format
DISPLAY PCIE,PFID=D1
IQP024I 07.47.23 DISPLAY PCIE 257
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000D1 Hardware Accelerator ALLC 0012 FPGHWAM 0140 0002
CLIENT ASIDS: NONE
Application Description: zEDC Express
Device State: Ready
Adapter Info - Relid: 00000B Arch Level: 03
Build Date: 02/26/2014 Build Count: 00
Application Info - Relid: 000000 Arch Level: 02
The DISPLAY PCIE,PFID=D1 command that is shown in Example 10-5 displays the status of the zEDC Express feature as up and ready for use.
By design, when you must configure the feature to be offline, you must use the FORCE option because zEDC Express feature is always allocated by the FPGHWAM address space. Example 10-6 shows the configure offline command that uses the force option.
Example 10-6 Configure offline using force
CONFIG PFID(D1),OFFLINE,FORCE
IEE505I PFID(D1),OFFLINE
IEE712I CONFIG PROCESSING COMPLETE
Displaying the PCIe status after the CONFIG PFID(xx),OFFLINE,FORCE command now shows the feature as STANDBY (see Example 10-7).
Example 10-7 Status after configure offline using force
DISPLAY PCIE
IQP022I 08.01.38 DISPLAY PCIE 284
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000A2 RoCE Express2 CNFG 0100 0003 1
000000A3 RoCE Express2 CNFG 0100 0004 2
00000102 8GB zHyperLink CNFG 0108 0003 1
00000103 8GB zHyperLink CNFG 0108 0004 2
000000D1 Hardware Accelerator STNBY 0140 0002
000000C1 Hardware Accelerator STNBY 0120 0002
10.3 Virtual Flash Memory
This section describes the configuration of the Virtual Flash Memory (VFM) feature on an IBM z14 ZR1.
10.3.1 VFM overview
The VFM is the replacement for the Flash Express features that were available on the zEC12, zBC12, z13, and z13s. VFM offers up to 2.0 TB of virtual flash memory in 512 GB increments. No application changes are required to change from Flash Express to VFM.
VFM is designed to help improve availability and handling of paging workload spikes when running z/OS V2.1, V2.2, or V2.3, or on z/OS V1.13. With this support, z/OS is designed to help improve system availability and responsiveness by using VFM across transitional workload events, such as market openings, and diagnostic data collection. z/OS is also designed to help improve processor performance by supporting middleware exploitation of pageable large (1 MB) pages.
Using VFM can help availability by reducing latency from paging delays that can occur at the start of the workday or during other transitional periods. It is also designed to help eliminate delays that can occur when collecting diagnostic data during failures. VFM can also be used in coupling facility images to provide extended capacity and availability for workloads that use IBM WebSphere MQ Shared Queues structures.
VFM can help organizations meet their most demanding service level agreements and compete more effectively. VFM is easy to configure, and provide rapid time to value.
For more information about the VFM feature, see the IBM z14 Technical Guide, SG24-8451.
10.3.2 Planning for VFM configuration
For planning considerations, see “Planning considerations for Virtual Flash Memory” on page 19.
10.3.3 Configuring VFM
The Assignment of VFM to LPARs is exclusively done with the definitions in the image activation profiles.
|
Note: Unlike the Flash Express cards, the allocation of VFM to LPARs cannot be altered to an activated LPAR. Therefore, the Manage Flash Allocation selection on the HMC is not supported for z14 ZR1.
|
Be aware of the following considerations when you allocate Virtual Flash Memory to a partition:
•When an allocation is first defined, you must set the initial and maximum allocation in 16 GB increments.
•A storage-class memory (SCM) allocation is put online to the z/OS image that is assigned to the partition at IPL time, unless the z/OS image is configured not to do so.
•z/OS allows more memory to be configured online, up to the maximum GB that is defined in this window, or up the maximum VFM available and not used by other LPARs.
•Minimum amounts are allocated from the available pool, so they cannot be overallocated.
•Maximum amounts can be overallocated up to the VFM LICCC value of the z14 ZR1.
•Maximum amounts must be greater than or equal to the initial amounts.
To allocate VFM to a partition, select the LPAR on the HMC and click Operational Customization → Customize/Delete Activation Profiles. Then, select the image profile and click Customize profile. The Initial and the Maximum values for the VFM are specified on the Storage tab. This configuration is shown in Figure 10-40.
Figure 10-40 Virtual Flash Memory Allocation in Image Profile
In this example, the initial value is set to 16 GB, and the maximum value is set to 32 GB. The z14 ZR1 features two VFM features installed, which allows a maximum of 1024 GB allocated to the LPAR.
These definitions do not change the settings of a running LPAR. They are used only at the activation of the LPAR. A newly activated LPAR comes up with the specified amount of initial memory VFM.
If in the image activation profile of an LPAR an amount of initial VFM storage is specified that is greater than the available amount of deallocated VFM on the z14 ZR1, the activation of this LPAR fails with message ACTZ01EB, as shown in Figure 10-41.
Figure 10-41 Insufficient VFM available, ACTZ01EB
|
Note: For a Coupling Facility LPAR, it is also possible to define an initial value and a higher maximum value for VFM in the image profile. However, it does not make sense to set the maximum value higher than the initial value because the CFCC does not support any command to set any reserved memory online.
|
10.3.4 VFM management
The memory allocation of a z14 ZR1 is shown on the SE in the Storage Information window. To view it, click the server and select Operational Customization → Storage Information. Then, the Base System Storage™ Allocation window is displayed, as shown in Figure 10-42.
Figure 10-42 Storage Information: Base System Storage Allocation
The z14 ZR1 server in our examples (MUSCA) includes 1024 GB installed, of which 64 GB is allocated to activated LPARs.
The Logical Partition Storage Allocation window shows the VFM allocation of the LPARs. For every LPAR, the initial and the maximum amount of VFM is listed (which were specified in the image activation profile), and also the currently allocated amount, as shown in Figure 10-43.
Figure 10-43 Storage Information: Logical Partition Storage Allocation
In z/OS for Flash Express and VFM, the PAGESCM parameter is supported in IEASYSxx. The syntax is shown in Example 10-8. This parameter determines whether and how much storage-class memory (SCM) is made available to an LPAR at IPL time.
Example 10-8 PAGESCM parameter
PAGESCM={xxxxxxM }
{xxxxxxG }
{xxT }
{ALL }
{NONE }
{0 }
This parameter specifies the minimum amount of SCM that should be made available for use as auxiliary storage. The system reserves this amount of SCM during IPL for subsequent use as auxiliary storage. More SCM is allocated on an as-needed basis if use of this initial amount of SCM is exceeded.
You can specify the following value ranges for the PAGESCM parameter to reserve SCM for paging at IPL:
xxxxxxM Specifies the amount of SCM to reserve for paging at IPL, in megabytes. This value can be 1 - 6 decimal digits.
xxxxxxG Specifies the amount of SCM to reserve for paging at IPL, in gigabytes. This value can be 1 - 6 decimal digits.
xxT Specifies the amount of SCM to reserve for paging at IPL, in terabytes. This value can be 1 - 2 decimal digits. The maximum amount of SCM supported for paging is 16 TB.
ALL Reserves all SCM for paging at IPL.
NONE SCM is not used for paging. This parameter remains in effect until the next IPL.
0 | 0M | 0G | 0T Indicates that no SCM is reserved for paging at IPL. Instead, SCM is allocated as needed, based on paging demand.
Default value ALL
Associated parmlib member None
The CONFIG SCM command is used to configure SCM online or offline to an LPAR (see Example 10-9).
Example 10-9 CONFIG SCM
CONFIG SCM(ddddddddM|G|T),ONLINE|ON
CONFIG SCM(ddddddddM|G|T),OFFLINE|OFF
CONFIG SCM(scm_ranges),OFFLINE|OFF
The system reconfigures SCM logically and physically. To bring SCM online, a number must be specified. To take SCM offline, a range of starting and ending addresses of the SCM blocks must be specified.
The command includes the following values:
ddddddddM|G|T The amount of SCM to be reconfigured. Specify up to eight decimal digits followed by a multiplier (M=megabytes, G=gigabytes, or T=terabytes) for this amount. Check your processor configuration for the supported SCM increment sizes. The value for dddddddd must be a multiple of the SCM increment size (usually 2, 4, or 8), and cannot exceed 16T.
Instead of specifying a decimal amount, you can specify a hexadecimal amount, with or without a multiplier, in the following format:
X'xxxxxx'
For example:
X'123456789A00000'
X'123'M
You can use underscore characters in any hexadecimal specification for more clarity. Underscore characters in the specification are ignored during processing.
|
Attention: If you take SCM offline and do not specify one or more scm_ranges, the system selects which SCM increments to take offline.
|
ONLINE or ON The system brings the specified amount of SCM online. ONLINE is the default value if only CONFIG SCM is specified. The system rejects the command if you specify the following values:
– A value that is not a multiple of the SCM increment size.
– A value that exceeds the total amount of SCM that is defined to this partition.
– A value that is not a valid amount of SCM (0, for example).
SCM is not supported or not defined on the system.
OFFLINE or OFF The system takes the specified amount or specified ranges of SCM offline.
|
Attention: Taking SCM offline can affect data reliability and performance. Consider the following implications before taking SCM offline:
•Your system must have enough auxiliary storage, which can include SCM and must include page data sets, to back critical system data. The CONFIG SCM OFFLINE command fails if taking the specified amount of SCM offline results in leaving auxiliary storage more than 50% full.
•SCM is used for paging critical address spaces and common address spaces. An insufficient amount of SCM causes those address spaces to page data sets, which can lead to a loss of critical data during a DASD IBM HyperSwap® scenario.
•SCM is used for paging large pages. If not enough SCM exists, 1 MB large pages are demoted to 256 4-KB pages and page-to-page data sets, which can negatively affect system performance.
|
scm_ranges Specifies a range of SCM or a list of ranges separated by commas identified by dddM|G|T-dddM|G|T; for example, 0G-16G, 32G-64G. The starting and ending addresses for each range of SCM must be multiples of the increment size.
The DISPLAY ASM and DISPLAY M commands include the following enhancements to display information and status that are related to Virtual Flash Memory:
DISPLAY ASM Lists SCM status along with paging data set status.
DISPLAY ASM,SCM Displays a summary of SCM usage.
DISPLAY M=SCM Displays SCM online/offline and increment information.
DISPLAY M=SCM(DETAIL) Displays detailed increment-level information.
|
Tip: You might notice a difference in usage numbers between the DISPLAY M=SCM and DISPLAY ASM commands. The difference is the result of how ASM perceives its use of the cache of available SCM block IDs that ASM maintains. To ASM, some block IDs are not in use because they were not yet assigned to page out requests. However, to the DISPLAY M=SCM command processor, block IDs are in use because they were assigned to ASM for its use.
|
VFM storage can also be used by coupling facility LPARs running CFCC Level 22 on z14 ZR1, which is similar to Flash Express that be used by coupling facility LPARs running CFCC Level 19 (on zEC12 and zBC12) or CFCC Level 20 or 21 (on z13 an z13s). Systems without this support cannot connect to or rebuild a structure by using SCM storage.
In 10.3.3, “Configuring VFM” on page 247, we allocated an initial VFM of 16 GB to the LPAR MUSCA11, and a maximum VFM of 32 GB (see Figure 10-40 on page 248). Now, from MUSCA11 running the z/OS image SC03, we issue the IBM MVS™ DISPLAY IPLINFO,PAGESCM command. Example 10-10 shows the results.
Example 10-10 DISPLAY IPLINFO,PAGESCM
DISPLAY IPLINFO,PAGESCM
IEE255I SYSTEM PARAMETER 'PAGESCM': NOT_SPECIFIED
Because no PAGESCM parameter is specified, the default value of ALL is used. If a VFM allocation is defined for the LPAR and PAGESCM=ALL is specified (or kept at the default), the initial amount of VFM that is specified is used automatically by z/OS for paging at IPL time. Likewise, if a specific amount is specified, this amount is made available for paging.
From SC03, run the enhanced DISPLAY ASM and DISPLAY M commands to display Virtual Flash Memory SCM-related information and status. The result for each command is shown in Example 10-11.
Example 10-11 Display commands
DISPPLAY ASM
IEE200I 12.04.41 DISPLAY ASM 799
TYPE FULL STAT DEV DATASET NAME
PLPA 28% OK 9A0B PAGE.SC03.PLPA
COMMON 0% OK 9A0B PAGE.SC03.COMMON
LOCAL 0% OK 9A36 PAGE.SC03.LOCAL01
LOCAL 0% OK 9AB6 PAGE.SC03.LOCAL02
LOCAL 0% OK 9B36 PAGE.SC03.LOCAL03
SCM 0% OK N/A N/A
PAGEDEL COMMAND IS NOT ACTIVE
DISPLAY ASM,SCM
IEE207I 12.05.26 DISPLAY ASM 801
STATUS FULL SIZE USED IN-ERROR
IN-USE 0% 4,194,304 20,247 0
DISPLAY M=SCM
IEE174I 12.06.04 DISPLAY M 803
STORAGE-CLASS MEMORY STATUS
32G DEFINED
ONLINE
0G-16G
16G OFFLINE-AVAILABLE
1% IN USE
SCM INCREMENT SIZE IS 16G
DISPLAY M=SCM(DETAIL)
IEE174I 12.06.53 DISPLAY M 805
STORAGE-CLASS MEMORY STATUS - INCREMENT DETAIL
32G DEFINED
ADDRESS IN USE STATUS
0G 1% ONLINE
ONLINE: 16G OFFLINE-AVAILABLE: 16G PENDING OFFLINE: 0G
1% IN USE
SCM INCREMENT SIZE IS 16G
From these commands, you see that 32 GB of VFM is defined, but only 16GB is online, while the other 16 GB are offline-available.
To vary another 16 GB VFM online to the example LPAR, issue the CONFIG SCM(xxG),ONLINE command, as shown in Example 10-12. The amount of VFM configured online must be specified according to the supported increment size. From these displays, the supported increment size is 16G.
Example 10-12 CONFIG SCM(16G),ONLINE
CONFIG SCM(16G),ONLINE
IEE195I SCM LOCATIONS 16G TO 32G ONLINE
IEE712I CONFIG PROCESSING COMPLETE
Issue the DISPLAY ASM and DISPLAY M commands again to display the status of the VFM and see that the 16 GB extra value is now online and available (see Example 10-13).
Example 10-13 Post configuration displays
DISPLAY ASM
IEE200I 12.10.14 DISPLAY ASM 845
TYPE FULL STAT DEV DATASET NAME
PLPA 28% OK 9A0B PAGE.SC03.PLPA
COMMON 0% OK 9A0B PAGE.SC03.COMMON
LOCAL 0% OK 9A36 PAGE.SC03.LOCAL01
LOCAL 0% OK 9AB6 PAGE.SC03.LOCAL02
LOCAL 0% OK 9B36 PAGE.SC03.LOCAL03
SCM 0% OK N/A N/A
PAGEDEL COMMAND IS NOT ACTIVE
DISPLAY ASM,SCM
IEE207I 12.10.41 DISPLAY ASM 847
STATUS FULL SIZE USED IN-ERROR
IN-USE 0% 8,388,608 20,247 0
DISPLAY M=SCM
IEE174I 12.08.47 DISPLAY M 843
STORAGE-CLASS MEMORY STATUS
32G DEFINED
ONLINE
0G-32G
0G OFFLINE-AVAILABLE
0% IN USE
SCM INCREMENT SIZE IS 16G
DISPLAY M=SCM(DETAIL)
IEE174I 12.11.46 DISPLAY M 849
STORAGE-CLASS MEMORY STATUS - INCREMENT DETAIL
32G DEFINED
ADDRESS IN USE STATUS
0G 1% ONLINE
16G 0% ONLINE
ONLINE: 32G OFFLINE-AVAILABLE: 0G PENDING OFFLINE: 0G
0% IN USE
SCM INCREMENT SIZE IS 16G
When displaying the Storage Information windows on the SE again (compare to Figure 10-42 on page 249 and Figure 10-43 on page 250), this change in LPAR MUSCA11 is reflected.
As shown in Figure 10-44, the amount of allocated VFM went up to 80 GB (compared to the 64 GB that is shown in Figure 10-42 on page 249).
Figure 10-44 Base System Storage Allocation
In Figure 10-45, the amount of VFM that is allocated to LPAR MUSCA11 went up to 32 GB.
Figure 10-45 Logical Partition Storage Allocation
You also can set VFM offline, even to an amount that is lower than the initial value that is specified in the image activation profile. If for LPAR MUSCA11 the amount of online VFM is reduced to 0 GB by issuing CONFIG SCM(32G),OFFLINE, this process results in the Storage Information windows that are displayed in Figure 10-46 on page 256 and Figure 10-47 on page 256.
In Figure 10-46, the amount of allocated VFM was reduced to 48 GB.
Figure 10-46 Base System Storage Allocation
In Figure 10-47, the amount of VFM that is allocated to LPAR MUSCA11 went down to 0 GB, which is lower than the initial 16 GB.
Figure 10-47 Logical Partition Storage Allocation
|
Note: An LPAR uses only the amount of VFM that is activated for that LPAR. VFM that is set offline by the operating system is returned to be used by other LPARs.
|
The allocation of VFM to a coupling facility LPAR is done in the same way as for z/OS LPARs and is described in 10.3.3, “Configuring VFM” on page 247. The amount of SCM that is allocated to a coupling facility LPAR can be displayed in the Operating System Messages window at the HMC.
For example, LPAR MUSCA1F, which allocated 32 GB of initial VFM storage, includes a message that shows the amount of SCM available, as shown in Example 10-14.
Example 10-14 CFCC messages with SCM
CF0280I CFCC Release 22.00, Service Level 00.30
Built on 01/31/2018 at 15:50:00
Code Load Features:
Facility Operational Level: 22
CF0011I Coupling Facility is active with:
2 CPs
2 CF Receiver Channels
0 CF Sender Channels
1559 MB of allocatable storage
32768 MB of Total SCM storage
The CF must know the algorithm of how the structure is used by the application. Currently, this algorithm is defined only for IBM MQ shared queues. To use this function, assign flash memory to your coupling facilities according to the procedure that is described next and update your structure definitions in your CFRM policy with the new parameter SCMMAXSIZE and SCMALGORITHM. For more information, see z/OS MVS Setting Up a Sysplex, SA23-1399.
IBM MQ for z/OS Version 7 or later allows the migration of IBM MQ shared queue objects to flash memory when structure usage exceeds the defined threshold. The IBM MQ objects are fetched back to real CF Storage when requested. This process provides an overflow capability for IBM MQ shared queues to handle workload peaks.
IBM RMF provides measurement data and reporting capabilities for VFM and Flash Express. The support enhances RMF Postprocessor and Monitor III reports with various new CF SCM statistics.
Coupling Facility SCM statistics are provided in the following reports:
•RMF Postprocessor Coupling Facility Activity (CF) report
•RMF Monitor III Coupling Facility Overview (CFOVER) report
•RMF Monitor III Coupling Facility Activity (CFACT) report
10.4 Shared Memory Communications over RDMA (SMC-R)
This section describes the configuration of the Shared Memory Communications over RDMA (SMC-R) that uses the 10GbE RoCE Express2 feature on an IBM z14 ZR1.
10.4.1 SMC-R overview
SMC-R uses the following IBM Z and industry standard communications technology:
•RDMA, which is based on queue pair (QP) technology that also uses an InfiniBand transport service type that is called reliable connected QPs (RC-QPs), which provide the following features:
– Represent SMC Links in a logical point-to-point connection.
– Transport data over unique RDMA network interface cards (RNICs) that are logically bound together to form Link Groups. Link Groups are used for high availability and load balancing needs.
Ports in the IBM Z 10GbE RoCE Express2 feature (also referred to as RNICs) are used as the physical transport layer for RDMA.
•Single root I/O virtualization (SR-IOV) is a Peripheral Component Interconnect® Express (PCIe) standard that define extensions to PCIe specifications. SR-IOV enables sharing of 10GbE RoCE Express2 ports between LPARs in the z14 and z14 ZR1.
For more information about the 10GbE RoCE Express2 feature and SMC-R, see IBM z14 Technical Guide, SG24-8451, and IBM z/OS V2R2 Communications Server TCP/IP Implementation Volume 1, SG24-8360.
10.4.2 Planning for SMC-R configuration
For an overview of planning considerations, see “Shared Memory Communications - RDMA” on page 22.
10.4.3 Configuring SMC-R
The 10GbE RoCE Express and 10GbE RoCE Express2 features are native PCIe features; therefore, the following HCD and IOCP definition rules differ from a non-native PCIe card, such as OSA Express:
•PCIe Function Identifier (FID) must be defined in HCD or HCM to create IOCP input:
– FID is a hexadecimal value (three heximal digits, range 000 - FFF), which specifies the PCIe function.
– It cannot be assigned to a channel subsystem, so that any LPAR can be defined to a function.
– It features a PARTITION parameter that dedicates it to one LPAR or allows reconfiguration among a group of LPARs. A function cannot be defined as shared.
– In z/OS system commands, PCIe FID is represented as PFID.
•If the intended PCIe hardware supports multiple partitions, it has a decimal Virtual Function Identifier (VF=) in the range 1 - n, where n is the maximum number of partitions that the PCIe feature supports.
•Other parameters that are specific to the PCIe feature. For example, the 10GbE RoCE Express requires a Physical Network Identifier (PNETID=), and the new 10GbE RoCE Express2 feature supports a port identifier (PORT=).
•For function mapping to hardware, assign a Physical Channel Identifier (PCHID=) to identify the hardware feature in a specific PCIe I/O drawer and the slot to be used for the defined function. The following methods can be used:
– Manually, by using the configurator (eCONFIG) PCHID report.
– By using the CHPID Mapping tool and the eConfig Configuration Report File (CFR) input.
|
Note: Unlike CHPIDs, multiple functions can be mapped to the same PCHID. This mapping is conceptually similar to mapping multiple InfiniBand coupling CHPIDs to the same adapter and port.
|
For more information about configuring an SMC-R connection by using HCD, see 15.2.3, “Defining a RoCE-2 PCIe function” on page 375.
10.4.4 SMC-R Management
This section introduces the z/OS commands that are related to the RoCE Express PCIe feature and shows the responses on our test system.
DISPLAY PCIE
You can use the DISPLAY PCIE command to display the following items:
•All registered device drivers (with assigned printable names).
•All available or in-use PCIe functions and their associated device types.
•Information about a specific PCIe device with a list of the client address spaces that use the device.
Example 10-15 shows an example of the DISPLAY PCIE command. You can confirm the FID and VFID that you defined. FID is represented as PFID (PCIE function identifiers).
Example 10-15 Example of D PCIE command
DISPLAY PCIE
IQP022I 13.32.12 DISPLAY PCIE 957
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000A2 RoCE Express2 CNFG 0100 0003 1
000000A3 RoCE Express2 CNFG 0100 0004 2
00000102 8GB zHyperLink CNFG 0108 0003 1
00000103 8GB zHyperLink CNFG 0108 0004 2
000000C1 Hardware Accelerator ALLC 0012 FPGHWAM 0120 0002
000000D1 Hardware Accelerator ALLC 0012 FPGHWAM 0140 0002
00000202 8GB zHyperLink CNFG 0168 0003 1
00000203 8GB zHyperLink CNFG 0168 0004 2
000000B2 RoCE Express2 CNFG 0160 0003 1
000000B3 RoCE Express2 CNFG 0160 0004 2
Example 10-16 shows an example of the DISPLAY PCIE,PFID=pfid command. After you define the new PCIe function, enter this command and confirm that its status is ACTIVE.
Example 10-16 Example of DISPLAY PCIE,PFID=pfid command
DISPLAY PCIE,PFID=0A3
IQP024I 13.36.48 DISPLAY PCIE 960
PCIE 0011 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
000000A3 RoCE Express2 CNFG 0100 0004 2
CLIENT ASIDS: NONE
PNetID 1: PERFNET
Example 10-17 shows an example of the DISPLAY PCIE,DD command. You can confirm the details of the device drives that are installed in the system.
Example 10-17 Example of DISPLAY PCIE,DD command
DISPLAY PCIE,DD
IQP023I 13.42.11 DISPLAY PCIE 962
PCIE 0011 ACTIVE
DEV TYPE DEVICE TYPE NAME
1014044B Hardware Accelerator
10140613 8GB zHyperLink
15B36750 10GbE RoCE
15B31003 10GbE RoCE
15B31004 10GbE RoCE Express
15B31016 RoCE Express2
15B31014 40GbE RoCE Express2
101404ED ISM
CONFIG command
You can use the CONFIG command to bring the PCIE function ID (PFID) online or offline.
Example 10-18 shows an example of CONFIG PFID(xx),ONLINE command.
Example 10-18 Example of CONFIG PFID(xx),ONLINE command
CONFIG PFID(A3),ONLINE
IEE504I PFID(A3),ONLINE
IEE712I CONFIG PROCESSING COMPLETE
Example 10-19 Example of CF PFID(x),OFFLINE command
CONFIG PFID(A3),OFFLINE
IEE505I PFID(A3),OFFLINE
IEE712I CONFIG PROCESSING COMPLETE
For more information about how to manage a RoCE Express feature, see IBM z/OS V2R2 Communications Server TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing, SG24-8360.
10.5 Shared Memory Communications - Direct Memory Access
This section describes the configuration of the Shared Memory Communiction - Direct Memory Access (SMC-D) connections on an IBM z14 ZR1.
10.5.1 SMC-D overview
SMC-D is a protocol that allows TCP socket applications to transparently use Internal Shared Memory (ISM). ISM is a virtual channel that is similar to IQD for HiperSockets. A virtual adapter is created in each z/OS LPAR and by using the SMC protocol, the memory is logically shared. The virtual network is provided by firmware.
SMC-R requires a TCP/IP connection and preserves the entire network infrastructure. SMC-D is also a “hybrid” solution. It uses a TCP connection to establish the SMC-D connection. The TCP path can be through an OSA-Express port or HiperSockets connection.
A TCP option (called SMCD) controls switching from TCP to “out of band” SMC-D. The SMC-D information is exchanged within the TCP data stream. Socket application data is exchanged through ISM (write operations). The TCP connection remains established to control the SMC-D connection.
For more information about SMC-D, see IBM z14 Technical Guide, SG24-8451, and IBM z/OS V2R2 Communications Server TCP/IP Implementation Volume 1, SG24-8360.
10.5.2 Planning for SMC-D configuration
For more information about planning considerations for SMC-D, see “Shared Memory Communications - RDMA” on page 22.
10.5.3 Configuring SMC-D
For more information about configuring an SMC-D connection by using HCD, see 15.2.2, “Defining an ISM PCIe function” on page 372.
10.5.4 SMC-D management
From an operational standpoint, SMC-D is similar to SMC-R. However, SMC-D uses direct memory access (DMA) instead of an RDMA. It also uses a virtual PCI adapter that is called ISM rather than an RNIC. The ISM interfaces are associated with IP interfaces (for example, HiperSockets or OSA-Express) and are dynamically created, automatically started and stopped, and auto-discovered.
SMC-D over ISM does not use QP technology like SMC-R. Therefore, links and Link Groups that are based on QPs (or other hardware constructs) are not applicable to ISM. SMC-D protocol features a design concept of a “logical point-to-point connection” called an SMC-D link.
|
Note: The SMC-D information in the netstat command displays is related to ISM link information (not Link Groups).
|
Internal Shared Memory technology
ISM is a virtual PCI network adapter that enables direct access to shared virtual memory, which provides highly optimized network communications for operating systems within the same IBM Z platform.
Virtual memory is managed by each z/OS (similar to SMC-R logically shared memory) following the existing IBM Z PCIe I/O translation architecture.
For more information about management of SMC-D, see IBM z/OS V2R2 Communications Server TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing, SG24-8360.
10.6 IBM zHyperlink Express
This section describes the configuration of the zHyperlink Express feature on an IBM z14 ZR1.
10.6.1 IBM zHyperlink Express overview
IBM zHyperLink Express is a new, short distance, IBM Z I/O channel that is designed for up to 10x lower latency than High-Performance FICON. zHyperLink is intended to speed IBM Db2® for z/OS transaction processing and improve active log throughput. This feature is in the PCIe I/O drawer and is a two-port adapter used for short distance, direct connectivity between a z14 ZR1 and a DS8880. It uses PCIe Gen3 technology, with x16 lanes that are bifurcated into x8 lanes for storage connectivity.
The zHyperLink Express is designed to drive distances up to 150 meters (492.1 feet) and support a link data rate of 8 GigaBytes per second (GBps). A zHyperlink port is fully sharable between all partitions because 127 virtual functions/PFIDs per link are supported.
IBM zHyperLink dramatically reduces the latency of DASD I/Os by interconnecting the z14 ZR1 CPC directly to the I/O Bay of the DS8880. This feature improves the application response time without application changes. zHyperLink is fast enough to run I/Os synchronously, so that the CPU can wait for the data, which results in the following advantages:
•No Undispatch of the running task
•No CPU Queueing Delays to resume it
•No host CPU cache disruption
•Reduced I/O service time
The zHyperLInk Express adapter takes one slot on z14 ZR1 PCIe I/O drawer, and each adapter has a single PCHID with two ports. Up to 16 zHyperLink Express adapters can be installed in one z14 ZR1 server, thus resulting in up to 32 links.
FICON connectivity to each storage system is still required to be used for the following purposes:
•Initialization of the zHyperlink connection
•I/Os that are not eligible for zHyperlink
•Failback when a zHyperlink request fails (for example, cache miss or busy condition)
For more information about the zHyperlink feature, see IBM z14 Technical Guide, SG24-8451, and IBM Z Connectivity Handbook, SG24-5444.
10.6.2 Planning for zHyperlink Express configuration
For more information about planning considerations, see 2.8.4, “Defining the IBM zHyperLink Express” on page 21.
10.6.3 Configuring zHyperlink Express
Like the 10GbE RoCE Express and the 10GbE RoCE Express2 features, the zHyperlink Express card is a native PCIe I/O feature. Therefore, the HCD and IOCP definition rules for native PCIe cards also apply, as described in “Configuring SMC-R” on page 258.
zHyperLink ports are defined as PCI functions in the I/O configuration, where the PCHID represents the card and the port number represents the port. Each zHyperlink can be shared by up to 127 virtual functions. Generally, define four PFIDs per zHyperLink port per LPAR so that up to four simultaneous operations can be active on a link at a time (more operations cause link busy type conditions). Also, eight FICON CHPIDs can still be defined per Logical Control Unit (LCU) because zHyperLink does not reduce that number.
The association between zHyperLinks and storage systems is automatically discovered; therefore, no I/O configuration definition is required. The z14 ZR1 firmware discovers the storage subsystem during link initialization, and z/OS associates the zHyperlinks with the devices at IPL or vary online time.
For more information about configuring a zHyperling Express card by using HCD, see 15.2.5, “Defining a zHyperLink PCIe function” on page 383.
Managing zHyperlink Express
To enable IBM DB2® to use zHyperlinks, the DB2 zParm form zHyperlink must be enabled, as shown in Figure 10-48.
|
DSNIIPA3 INSTALL DB2 - DATA PARAMETERS PANEL
===>
Check parameters and reenter to change:
1 PERMANENT UNIT NAME ==> 3390 Device type for MVS catalog and
partitioned data sets
2 TEMPORARY UNIT NAME ==> SYSDA Device type for temporary data sets
3 DB2 zHyperlinks SCOPE ==> ENABLE Scope of zHyperlinks I/O connections:
(ENABLE, DISABLE, DATABASE, LOG)
---------------- SMS -----------------
VOL/SER DATA CLASS MGMT CLASS STOR CLASS
------- ---------- ---------- ----------
4 CLIST ALLOCATION ==> ______ ==> ________ ==> ________ ==> ________
5 NON-VSAM DATA ==> ______ ==> ________ ==> ________ ==> ________
6 VSAM CATALOG, DEFAULT, ==> ______ ==> ________ ==> ________ ==> ________
AND WORK FILE DATABASE
7 LOG COPY 1, BSDS 2 ==> ______ ==> ________ ==> ________ ==> ________
8 LOG COPY 2, BSDS 1 ==> ______ ==> ________ ==> ________ ==> ________
PRESS: ENTER to continue RETURN to exit HELP for more information
|
Figure 10-48 DB2 zParm: zHyperlink
The acceptable values for the DB2 zHyperlink Scope are:
ENABLE DB2 requests the zHyperlink protocol for all eligible I/O requests
DISBALE DB2 does not use the zHyperlink for any I/O requests
DATABASE DB2 requests the zHyperlink protocol for only database synchronous read I/Os
LOG DB2 requests the zHyperlink protocol for only log write I/Os
At GA time, zHyperlink is used by synchronous DB2 database reads only, which are provided with APAR PI77461 for DB2 V12.
To enable z/OS for zHyperlink read I/Os, the ZHYPERLINK OPER=READ statement must be added to the IECIOSxx parmlib member, as shown in Example 10-20.
Example 10-20 IECIOSxx parmlib enabled for zHyperlink read I/Os
VIEW SYS1.PARMLIB(IECIOSFC) - 01.19 Columns 00001 00072
Command ===> Scroll ===> PAGE
****** ***************************** Top of Data ******************************
000001 CTRACE(CTIIOS00)
000006 HYPERPAV=XPAV
000007 MIDAW=YES
000008 ZHPF=YES
000009 ZHYPERLINK OPER=READ
****** **************************** Bottom of Data ****************************
This process can also be done dynamically by entering the SETIOS ZHYPERLINK,OPER=READ console command. The corresponding display command is DISPLAY IOS,ZHYPERLINK, as shown in Example 10-21.
Example 10-21 DISPLAY IOS,ZHYPERLINK
DISPLAY IOS,ZHYPERLINK
IOS634I 14.20.06 IOS SYSTEM OPTION 998
ZHYPERLINK IS ENABLED FOR READ OPERATIONS
The DISPLAY PCIE command can be used to display the available PCIe function IDs for zHyperlink, as shown in Example 10-22.
Example 10-22 DISPLAY PCIE
DISPLAY PCIE
IQP022I 09.50.36 DISPLAY PCIE 291
PCIE 0010 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
00000001 Hardware Accelerator ALLC 0011 FPGHWAM 0118 0002
00000011 Hardware Accelerator ALLC 0011 FPGHWAM 0138 0002
00001304 8GB zHyperLink ALLC 0017 IOSAS 01BC 0005 1
00001305 8GB zHyperLink ALLC 0017 IOSAS 01BC 0006 1
00001306 8GB zHyperLink ALLC 0017 IOSAS 01BC 0007 1
00001307 8GB zHyperLink ALLC 0017 IOSAS 01BC 0008 1
00001384 8GB zHyperLink CNFG 01BC 0005 2
00001385 8GB zHyperLink CNFG 01BC 0006 2
00001386 8GB zHyperLink CNFG 01BC 0007 2
00001387 8GB zHyperLink CNFG 01BC 0008 2
00001004 8GB zHyperLink ALLC 0017 IOSAS 013C 0005 1
Example 10-23 shows the DISPLAY PCIE=pfid command to display a specific zHyperlink PCIe function ID.
Example 10-23 DiSPLAY PCIE=pfid
DISPLAY PCIE,PFID=1304
IQP024I 09.54.25 DISPLAY PCIE 299
PCIE 0010 ACTIVE
PFID DEVICE TYPE NAME STATUS ASID JOBNAME CHID VFN PN
00001304 8GB zHyperLink ALLC 0017 IOSAS 01BC 0005 1
CLIENT ASIDS: NONE
CU WWNN: 5005076306FFD680 CU Link Id: 0380
S/W State: Allocated
Port State: Operational
CU Node Descriptor: 002107.981.IBM.75.0000000FAT71
The results of issuing the DISPLAY M=CU(cun) command against a control unit enabled for zHyperlink are shown in Example 10-24.
Example 10-24 DISPLAY M=CU(cun)
DISPLAY M=CU(9000)
IEE174I 09.56.31 DISPLAY M 307
CONTROL UNIT 9000
CHP 8C 94 AC D6 B4 BC C4 CC
ENTRY LINK ADDRESS C535 C543 C343 20F7 C907 C903 C713 C74A
DEST LINK ADDRESS C330 C340 C530 C540 C730 C740 C930 C940
CHP PHYSICALLY ONLINE Y Y Y Y Y Y Y Y
PATH VALIDATED Y Y Y N Y N Y Y
MANAGED N N N N N N N N
ZHPF - CHPID Y Y Y Y Y Y Y Y
ZHPF - CU INTERFACE Y Y Y N Y N Y Y
INTERFACE ID 0010 0011 0012 .... 0140 .... 0142 0143
MAXIMUM MANAGED CHPID(S) ALLOWED = 0
DESTINATION CU LOGICAL ADDRESS = 00
CU ND = 002107.981.IBM.75.0000000FAT71.0010
CU NED = 002107.981.IBM.75.0000000FAT71.0000
TOKEN NED = 002107.900.IBM.75.0000000FAT71.0000
WWNN = 5005076306FFD680
FUNCTIONS ENABLED = ZHPF, ZHYPERLINK, XPAV
XPAV CU PEERS = 9000, 9200
DEFINED DEVICES
09000-0907F
DEFINED PAV ALIASES
19000-1907F
ZHYPERLINKS
PFID PCHID Port LinkId S/W St Port St
00001004 013C 01 0180 Alloc Oper
00001005 013C 01 0180 Alloc Oper
00001006 013C 01 0180 Alloc Oper
00001007 013C 01 0180 Alloc Oper
00001104 0178 01 0580 Alloc Oper
00001105 0178 01 0580 Alloc Oper
00001106 0178 01 0580 Alloc Oper
00001107 0178 01 0580 Alloc Oper
00001204 017C 01 0780 Alloc Oper
00001205 017C 01 0780 Alloc Oper
00001206 017C 01 0780 Alloc Oper
00001207 017C 01 0780 Alloc Oper
00001304 01BC 01 0380 Alloc Oper
00001305 01BC 01 0380 Alloc Oper
00001306 01BC 01 0380 Alloc Oper
00001307 01BC 01 0380 Alloc Oper
The results for the DISPLAY M=DEV(devno) command against a device enabled for zHyperlink are shown in Example 10-25.
Example 10-25 DISPLAY M=DEV(devno)
DISPLAY M=DEV(9000)
IEE174I 10.09.09 DISPLAY M 317
DEVICE 09000 STATUS=ONLINE
ENTRY LINK ADDRESS C535 C543 C343 20F7 C907 C903 C713 C74A
DEST LINK ADDRESS C330 C340 C530 C540 C730 C740 C930 C940
PATH ONLINE Y Y Y N Y N Y Y
CHP PHYSICALLY ONLINE Y Y Y Y Y Y Y Y
PATH OPERATIONAL Y Y Y N Y N Y Y
MANAGED N N N N N N N N
CU NUMBER 9000 9000 9000 9000 9000 9000 9000 9000
INTERFACE ID 0010 0011 0012 .... 0140 .... 0142 0143
MAXIMUM MANAGED CHPID(S) ALLOWED: 0
DESTINATION CU LOGICAL ADDRESS = 00
SCP CU ND = 002107.981.IBM.75.0000000FAT71.0010
SCP TOKEN NED = 002107.900.IBM.75.0000000FAT71.0000
SCP DEVICE NED = 002107.900.IBM.75.0000000FAT71.0000
WWNN = 5005076306FFD680
HYPERPAV ALIASES CONFIGURED = 128
ZHYPERLINKS AVAILABLE = 16
FUNCTIONS ENABLED = MIDAW, ZHPF, XPAV, ZHYPERLINK
This display command was enhanced with the new parameter ZHYPERLINK to show whether the device supports zHyperlink. The response is shown for a device that is capable for zHyperlink in Example 10-26.
Example 10-26 DISPLAY M=DEV(devno),ZHYPERLINK: Device enabled for zHyperlink
DISPLAY M=DEV(7000),ZHYPERLINK
IEE587I 14.57.37 DISPLAY M 356
DEVICE 07000 STATUS=ONLINE
DEVICE IS ENABLED FOR ZHYPERLINK
READ OPERATIONS ARE ENABLED
WRITE OPERATIONS ARE DISABLED FOR THE FOLLOWING REASON(S):
ZHYPERLINK WRITES ARE DISABLED FOR THE SYSTEM
CONTROL UNIT DOES NOT SUPPORT ZHYPERLINK WRITES
The result for a device that is not capable for zHyperlink is shown in Example 10-27.
Example 10-27 DISPLAY M=DEV(devno,ZHYPERLINK: Device not enabled for zHyperlink
DISPLAY M=DEV(265E),ZHYPERLINK
IEE587I 14.59.25 DISPLAY M 373
DEVICE 0265E STATUS=ONLINE
DEVICE IS DISABLED FOR ZHYPERLINK FOR THE FOLLOWING REASON(S):
CONTROL UNIT DOES NOT SUPPORT ZHYPERLINK
THERE ARE NO ZHYPERLINKS AVAILABLE
WRITE OPERATIONS ARE DISABLED FOR THE FOLLOWING REASON(S):
ZHYPERLINK WRITES ARE DISABLED FOR THE SYSTEM
1 Regional Crypto Enablement is also supported, such as on z13, but this topic is not covered here.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.