This chapter describes multipath SCSI support, and applies to disks and tapes except where noted. The SCSI protocol is used on both the parallel SCSI interconnect and the Fibre Channel interconnect. The term SCSI is used to refer to either parallel SCSI, or Fibre Channel (FC) devices throughout the chapter.

The following topics are presented in this chapter:

• Overview of multipath SCSI support ( Section 6.1)
• Configuration requirements and restrictions ( Section 6.2)
• HSx failover modes ( Section 6.3)
• Parallel SCSI multipath configurations (disks only) ( Section 6.4)
• Device naming for parallel SCSI multipath disk configurations ( Section 6.5)
• Fibre Channel multipath configurations ( Section 6.6)
• Implementing multipath configurations ( Section 6.7)

A multipath SCSI configuration provides failover from one path to a device to another path to the same device. Multiple paths to the same device increase the availability of that device for I/O operations. Multiple paths also offer higher aggregate performance. Figure 6-1 shows a multipath SCSI configuration. Two paths are configured from a computer to the same virtual storage device.

Multipath SCSI configurations for disk devices can use either parallel SCSI or Fibre Channel as the storage interconnect, as illustrated by Figure 6-1. Multipath SCSI configurations for tape devices can use only Fibre Channel as the storage interconnect.

Two or more paths to a single device are called a multipath set. When the system configures a path to a device, it checks for an existing device with the same name but a different path. If such a device is found, and multipath support is enabled, the system either forms a multipath set or adds the new path to an existing set. If multipath support is not enabled, then no more than one path to a device is configured.

The system presents a multipath set as a single device. The system selects one path to the device as the "current" path, and performs all I/O over this path until there is a failure or the system manager requests that the system switch to another path.

Multipath SCSI support provides the following types of failover:

• Direct SCSI to direct SCSI
• Direct SCSI to MSCP served (disks only)
• MSCP served to direct SCSI (disks only)

Direct SCSI to direct SCSI failover requires the use of multiported SCSI devices. Direct SCSI to MSCP served failover requires multiple hosts per SCSI bus, but does not require multiported SCSI devices. These two failover types can be combined. Each type and the combination of the two are described next.

6.1.1 Direct SCSI to Direct SCSI Failover

Direct SCSI to direct SCSI failover can be used on systems with multiported SCSI devices. The dual HSZ70, the HSZ80, the HSG80, the dual MDR, and the HSV110 are examples of multiported SCSI devices. A multiported SCSI device can be configured with multiple ports on the same physical interconnect so that if one of the ports fails, the host can continue to access the device through another port. This is known as transparent failover mode and has been supported by OpenVMS for disk devices since Version 6.2.

OpenVMS Version 7.2 introduced support for a new failover mode in which the multiported disk device can be configured with its ports on different physical interconnects. This is known as multibus failover mode.

The HSx failover modes are selected by HSx console commands. Transparent and multibus modes are described in more detail in Section 6.3.

Figure 6-1 is a generic illustration of a multibus failover configuration.

The two logical disk devices shown in Figure 6-1 represent virtual storage units that are presented to the host by the HSx controller modules. Each logical storage unit is "on line" to one of the two HSx controller modules at a time. When there are multiple logical units, they can be on line to different HSx controllers so that both HSx controllers can be active at the same time.

In transparent mode, a logical unit switches from one controller to the other when an HSx controller detects that the other controller is no longer functioning.

In multibus mode, as shown in Figure 6-1, a logical unit switches from one controller to the other when one of the following events occurs:

• One HSx controller detects that the other controller is no longer functioning.
• The OpenVMS multipath software detects that the current path has failed and issues a command to cause a switch.
• The OpenVMS system manager issues a command to cause a switch.

Figure 6-1 Multibus Failover Configuration

Note the following about Figure 6-1:

• Host has two adapters.
• Interconnects can both be parallel SCSI (HSZ70 or HSZ80) or both be Fibre Channel (HSGx or HSVx) but not mixed.
• Storage cabinet contains two HSx controllers configured for multibus failover mode.

The multibus configuration offers the following advantages over transparent failover:

• Higher aggregate performance with two host adapters and two HSx controller modules in operation.
• Higher availability because the storage is still accessible when a host adapter, the interconnect, or the HSx controller module on a path fails.

In a multibus failover configuration, SAS and its controllers (LSI 1068 and LSI 1068e) can also be used.

6.1.2 Direct SCS to MSCP Served Failover (Disks Only)

OpenVMS provides support for multiple hosts that share a SCSI bus. This is known as a multihost SCSI OpenVMS Cluster system. In this configuration, the SCSI bus is a shared storage interconnect. Cluster communication occurs over a second interconnect (LAN).

Multipath support in a multihost SCSI OpenVMS Cluster system enables failover from directly attached SCSI storage to MSCP served SCSI storage, as shown in Figure 6-2.

Figure 6-2 Direct SCSI to MSCP Served Configuration With One Interconnect

Note the following about this configuration:

• Two hosts are connected to a shared storage interconnect.
• Two hosts are connected by a second interconnect (LAN) for cluster communications.
• The storage devices can have a single port or multiple ports.
• If node Edgar's SCSI connection to the storage fails, the SCSI storage is MSCP served by the remaining host over the cluster interconnect.

Multipath support in such a multihost SCSI OpenVMS Cluster system also enables failover from MSCP served SCSI storage to directly attached SCSI storage. For example, the following sequence of events can occur on the configuration shown in Figure 6-2:

• Node POE is using node EDGAR as an MSCP server to access some storage device on the shared storage interconnect.
• On node EDGAR, the direct connection to the shared storage fails, or node EDGAR is shut down, or node EDGAR becomes unreachable via the cluster interconnect.
• Node POE switches to using its direct path to the shared storage.

In a direct SCSI to MSCP served failover configuration, SAS and its controllers (LSI 1068 and LSI 1068e) can also be used.

6.1.3 Configurations Combining Both Types of Multipath Failover

In a multihost SCSI OpenVMS cluster system, you can increase disk storage availability by configuring the cluster for both types of multipath failover (direct SCSI to direct SCSI and direct SCSI to MSCP served SCSI), as shown in Figure 6-3.

Figure 6-3 Direct SCSI to MSCP Served Configuration With Two Interconnects

Note the following about this configuration:

• Both nodes are directly connected to both storage interconnects.
• Both nodes are connected to a second interconnect for cluster communications.
• Each HSx storage controller is connected to only one interconnect.
• Both HSx storage controllers are in the same cabinet.

This configuration provides the advantages of both direct SCSI failover and direct to MSCP served failover.

In this type of configuration, SAS and its controllers (LSI 1068 and LSI 1068e) can also be used.

6.2 Configuration Requirements and Restrictions

Table 6-1 Multipath SCSI and FC Configuration Requirements

Component	Description
Host adapter	For parallel SCSI, the KZPBA-CB must be used. It is the only SCSI host adapter that supports disk multipath failover on OpenVMS. For Fibre Channel, both the KGPSA-BC and the KGPSA-CA support multipath failover on OpenVMS.
Alpha console firmware	For systems with HSZ70 and HSZ80, the minimum revision level is 5.3 or 5.4, depending on your AlphaServer. For systems with HSG80, the minimum revision level is 5.4
Controller firmware	For HSZ70, the minimum revision level is 7.3; for HSZ80, it is 8.3; for HSG80, it is 8.4. For MDR, the minimum revision level is 1170.
Controller module mode	Must be set to multibus mode (disks only). The selection is made at the HS x console.
Full connectivity	All hosts that are connected to an HS x in multibus mode must have a path to both HS x controller modules. This is because hosts that are connected exclusively to different controllers will switch the logical unit back and forth between controllers, preventing any I/O from executing. To prevent this from happening, always provide full connectivity from hosts to controller modules. If a host's connection to a controller fails, then take one of the following steps to avoid indefinite path switching: Repair the connection promptly. Prevent the other hosts from switching to the partially connected controller. This is done by either disabling switching to the paths that lead to the partially connected controller (see Section 6.7.11), or by shutting down the partially connected controller. Disconnect the partially connected host from both controllers.
Allocation classes	For parallel SCSI, a valid HSZ allocation class is required (see Section 6.5.3). If a SCSI bus is configured with HSZ controllers only, and all the controllers have a valid HSZ allocation class, then it is not necessary to adhere to the older SCSI device naming rules for that bus. That is, the adapters require neither a matching port allocation class nor a matching node allocation class and matching OpenVMS adapter device names. However, if there are non-HSZ devices on the bus, or HSZ controllers without an HSZ allocation class, you must follow the standard rules for shared SCSI buses for node and port allocation class assignments and for controller device names. Booting from devices with an HSZ allocation class is supported on all AlphaServers that support the KZPBA-CB except for the AlphaServer 2 x00(A). The controller allocation class is not used for FC devices. Note: If you are using Volume Shadowing for OpenVMS, every disk must have a nonzero allocation class. All FC disks attached to HSG and HSV controllers are automatically assigned the allocation class value of 1, which satisfies the volume shadowing requirement.

The restrictions for multipath FC and SCSI configurations are presented in Table 6-2.

Table 6-2 Multipath FC and SCSI Configuration Restrictions

Capability	Description
Devices supported	DKDRIVER disk devices attached to HSZ70, HSZ80, HSG60, HSG80, and HSV110 controller modules are supported. MKDRIVER tapes and GKDRIVER tape robots attached to an MDR are supported. Other device types, such as tapes, and generic class drivers, such as GKDRIVER, are not supported. Note that under heavy load, a host-initiated manual or automatic switch from one controller to another may fail on an HSZ70 or HSZ80 controller. Testing has shown this to occur infrequently. This problem has been fixed for the HSZ70 with the firmware HSOF V7.7 (and later). The problem will be fixed for the HSZ80 in a future release. This problem does not occur on HSG x or HSV x controllers, nor on the MDR.
Mixed-version and mixed-architecture clusters	All hosts that are connected to an HSZ, HSG, or HSV in multibus mode must be running OpenVMS Version 7.2 or higher. To use multipath failover to a served path, MPDEV_REMOTE must be enabled on all systems that have direct access to shared SCSI or Fibre Channel devices.
SCSI to MSCP failover MSCP to SCSI failover	Multiple hosts must be attached to SCSI disk devices via a shared SCSI bus (either parallel SCSI or Fibre Channel). The MPDEV_REMOTE system parameter must be set to 1 on these hosts.
Volume Shadowing for OpenVMS	The use of default settings for certain system parameters may lead to the occasional removal of shadow set members that are configured for multipath support. The shadow set members where this has been observed are using Volume Shadowing for OpenVMS. Therefore, when configuring multipath shadow sets using Volume Shadowing for OpenVMS, observe the following recommended setting for these system parameters: System Parameter Recommended Setting MSCP_CMD_TMO 60 as a minimum. The value of 60 is appropriate for most configurations. Some configurations may require a higher setting. SHADOW_MBR_TMO At least 3 x MSCP_CMD_TMO. SHADOW_SYS_TMO At least 3 x MSCP_CMD_TMO. MVTIMEOUT At least 4 x SHADOW_MBR_TMO.

6.3 HSx Failover Modes

The HSZ70, HSZ80, and HSGx implement two modes of failover operation when they are in a dual-redundant configuration, transparent failover mode, and multibus failover mode. HSVx supports multibus failover only.

For the system to operate correctly, the HSx failover mode must be compatible with the configuration of the interconnect hardware and the host operating system software, as described in the following sections.

6.3.1 Transparent Failover Mode

In transparent failover mode, the HSx presents each logical unit on one port of the dual controller pair. Different logical units may be assigned to different ports, but an individual logical unit is accessible through one port at a time. As shown in Figure 6-4, when the HSZ detects that a controller module has failed, it moves the logical unit to the corresponding port on the surviving controller.

Figure 6-4 Storage Subsystem in Transparent Mode

The assumption in transparent mode is that the two ports are on the same host bus, so the logical unit can move from one port to the other without requiring any changes to the host's view of the device. The system manager must ensure that the bus configuration is correct for this mode of failover. OpenVMS has supported transparent failover for HSZ controllers since Version 6.2.

To select transparent failover mode on the HSZ or HSG, enter one of the following commands at the console, depending on your configuration:

HSZ> SET FAILOVER COPY=THIS_CONTROLLER

or

HSZ> SET FAILOVER COPY=OTHER_CONTROLLER

An example of the output of a console SHOW command on an HSZ in transparent mode follows:

z70_A => SHOW THIS_CONTROLLER Controller: HSZ70 ZG64100160 Firmware XB32-0, Hardware CX25 Configured for dual-redundancy with ZG64100136 In dual-redundant configuration Device Port SCSI address 7 Time: 02-DEC-1998 09:22:09 Host port: SCSI target(s) (0, 2, 3, 4, 5, 6) Preferred target(s) (3, 5) TRANSFER_RATE_REQUESTED = 20MHZ Host Functionality Mode = A Allocation class 0 Command Console LUN is target 0, lun 1 Cache: 32 megabyte write cache, version 4 Cache is GOOD Battery is GOOD No unflushed data in cache CACHE_FLUSH_TIMER = DEFAULT (10 seconds) NOCACHE_UPS

6.3.2 Multibus Failover Mode (Disks Only)

In multibus failover mode, the HSx responds to SCSI Inquiry commands from the host on all ports of the dual controller pair. This allows the host to be aware of all the possible paths to the device. There are two advantages to having the host aware of all the paths to a device:

• The host can select an alternate path if it detects a failure on the current path. This is in addition to the failover that occurs when the HSx controller detects a failure, as is provided in transparent mode.
• The paths do not need to be on the same host bus. When the host is aware of the alternate paths, it can adjust its addressing methods appropriately to select a different path. This removes the SCSI bus as a single point of failure.

Note that, although the logical unit is visible on all ports, it is on line and thus capable of doing I/O on the ports of only one controller at a time. Different logical units may be on line to different controllers, but an individual logical unit is on line to only one controller at a time, as shown in Figure 6-5.

Figure 6-5 Storage Subsystem in Multibus Mode

You can determine which controller a logical unit is on line to by entering the HSx console command, as follows:

z70_A => SHOW UNIT FULL LUN Uses -------------------------------------------------------------- D200 DISK20300 Switches: RUN NOWRITE_PROTECT READ_CACHE MAXIMUM_CACHED_TRANSFER_SIZE = 32 ACCESS_ID = ALL State: ONLINE to the other controller PREFERRED_PATH = OTHER_CONTROLLER Size: 2050860 blocks

The host executes I/O to a logical unit on one path at a time, until that path fails. If a controller has two ports, then different hosts can access the same logical unit over different ports of the controller to which the logical unit is on line.

An HSx in multibus failover mode can only be used with the multipath functionality introduced in OpenVMS Version 7.2.

To select multibus failover mode, enter one of the following commands at the HSx: console, whichever is appropriate to your configuration:

HSZ> SET MULTIBUS_FAILOVER COPY=THIS_CONTROLLER

or

HSZ> SET MULTIBUS_FAILOVER COPY=OTHER_CONTROLLER

An example of the output of a console SHOW command on an HSx controller in multibus mode follows:

z70_B => SHOW THIS_CONTROLLER Controller: HSZ70 ZG64100136 Firmware XB32-0, Hardware CX25 Configured for MULTIBUS_FAILOVER with ZG64100160 In dual-redundant configuration Device Port SCSI address 6 Time: NOT SET Host port: SCSI target(s) (0, 2, 3, 4, 5, 6) TRANSFER_RATE_REQUESTED = 20MHZ Host Functionality Mode = A Allocation class 0 Command Console LUN is target 0, lun 1 Cache: 32 megabyte write cache, version 4 Cache is GOOD Battery is GOOD No unflushed data in cache CACHE_FLUSH_TIMER = DEFAULT (10 seconds) NOCACHE_UPS