Performance During Copy and Merge Operations
| | |
| ||
| ||
A shadow set is in a transient state when some or all of its members are undergoing a copy or merge operation. During merge operations, Volume Shadowing for OpenVMS ensures consistency by reading the data from one member and making sure it is the same as the data contained in the same LBNs on the other members of the shadow set. If the data is different, the shadowing software updates the LBN on all members before returning the I/O request. For copy operations, the shadowing software reads data from a source member and writes the data to the same LBN on target members.
At the same time it performs a merge or copy operation, the shadowing software continues to process application and user I/O requests. The I/O processing necessitated by a copy operation can result in decreased performance as compared with the possible performance of the same shadow set under steady-state conditions. However, if your shadow set members are configured on controllers that support the shadowing assisted copy and assisted merge operations, you can significantly improve the speed with which a shadow set performs a copy or merge operation. Volume Shadowing for OpenVMS supports both assisted and unassisted merge and copy operations.
The following list describes how the performance of a shadow set might be affected while an unassisted merge or copy operation is in progress. See Chapter 6 for a description of assisted copy and merge operations.
Copy operations
A copy operation is started on a two- or three-member shadow set either when you mount the shadow set to create it or to add a new member to an existing shadow set. During a copy operation, members that are targets of the operation cannot provide data availability until the operation completes. Therefore, the shadowing software performs the copy operation as quickly as possible to make the shadow set fully available.
During a copy operation, the shadowing software gives equal priority to user and application I/O requests and to I/O requests necessary to complete the copy operation. The performance of a shadow set during a copy operation is reduced because:
The shadowing software must follow special protocols for user read and write I/O requests during a copy operation.
Copy operation I/O requests are large in size and have the same priority as user and application I/O requests.
In addition, other system resources are utilized during a copy operation. Depending on the access path to the individual shadow set members, these resources could include the disk controller, interconnects, interconnect adapters, and systems.
Because you explicitly start copy operations when you mount a new shadow set or add members to an existing set, you can control when the shadowing software performs a copy operation. Therefore, you can minimize the effect on users and applications in the system by limiting the number of copy operations that occur at the same time. For example, when you create new sets or add new members, try to add the sets or members during periods of low system activity, and do not mount several sets at the same time.
You can further minimize the effect on users and applications in the system by using the minicopy operation, introduced with OpenVMS Version 7.3. The minicopy operation can significantly reduce the time it takes to return a shadow set member to shadow set. By the use of write bitmap technology, the minicopy operation copies only the data that was changed while the member was dismounted. For more information, see Chapter 7.
Merge operations
In contrast to copy operations, merge operations are not under the control of a user or program. The shadowing software automatically initiates a merge operation on a shadow set as a result of the failure of a node on which the shadow set is mounted.
As in the case of a copy operation, the volume shadowing software ensures that all I/O requests to the shadow set follow appropriate protocols to ensure data consistency. However, when a shadow set is undergoing a merge operation, full data availability exists in the sense that individual members of the set are valid data sources and are accessible by applications and users on the system. Therefore, it is not urgent for the shadowing software to finish the merge operation, especially when the system is being heavily used. Because of this major distinction from a copy operation, the shadowing software implicitly places a higher priority on user activity to the shadow set. Volume Shadowing for OpenVMS does this by detecting and evaluating system load, and then dynamically controlling or throttling the merge operation so that other I/O activity can proceed without interference.
Because the merge throttle slows merge operations when there is heavy application and user I/O activity on the system, the merge operation can take longer than copy operations. The merge throttle allows application and user activity to continue unimpeded by the merge operation when heavy work loads are detected.
On the other hand, the read performance of a shadow set during a merge operation is reduced because the shadowing software must perform data integrity checks on all members for every read request. The volume shadowing software reads the data from the same LBN on all members of the shadow set, compares the data, and repairs any inconsistencies before returning the read data to the user.
During an unassisted shadow set merge operation, read I/O performance available to applications is reduced by two factors:
The need to perform data consistency checks on all read I/Os
Contention for I/O bandwidth by the shadow set merge operation
The shadow set merge operation employs a throttling mechanism to limit the impact of merge I/O on applications. The merge process is throttled by introducing a delay between merge I/Os when system load is detected. The logic for computing this delay has been redesigned for OpenVMS Alpha Version 7.3-2.
Depending on the requirements of your application load, it may be desirable to minimize the impact of the merge I/O on your applications and allow the merge to take longer to complete; conversely, it may be desirable to make the merge complete quickly and accept the impact on applications. The following two parameters, specified with logical names, allow you to make this tradeoff for all shadow sets on your system:
SHAD$MERGE_DELAY_THRESHOLD specifies the threshold I/O time at which the merge process becomes throttled. The threshold is expressed as a multiplier on the “ideal” I/O time measured by the system on the shadow set. The default value of 200 is equivalent to a multiplier of 1. This parameter can be set to values from 0 to 20000.
SHAD$MERGE_DELAY_FACTOR specifies the length of the I/O delay. The I/O delay time is computed by subtracting the threshold from the currently observed merge I/O time. The delay factor acts as a divisor to the delay time; the default value of 200 is equivalent to a divisor of 1. This parameter can be set to values from 2 to 100000.
The delay between merge I/O operations is computed as follows:
delay-time = (current I/O time - ideal I/O time * MERGE_DELAY_THRESHOLD/200) * 200/MERGE_DELAY_FACTOR
Increasing either parameter causes merge operations to run faster and place a heavier load on the system; conversely, decreasing them causes merge operations to run more slowly. Setting the parameters to 200 or lower slows merge operations much more gradually than in previous OpenVMS versions.
In addition to the previous two logical names which specify the parameters for all shadow sets on your system, you can specify parameters for specific shadow sets (designated by "_DSAnnnn" with logical names of the form:
SHAD$MERGE_DELAY_THRESHOLD_DSAnnnn
SHAD$MERGE_DELAY_FACTOR_DSAnnnn
You can use the same ranges of values for these parameters that you use for SHAD$MERGE_DELAY_THRESHOLD and SHAD$MERGE_DELAY_FACTOR.
The applicable logical names are sampled by the shadow copy server every 1000 I/Os, so that an in-progress copy or merge responds to a parameter change after a short delay.
There are two types of performance assists: the merge assist and the copy assist. The merge assist improves performance by using information that is maintained in controller-based write logs to merge only the data that is inconsistent across a shadow set. When a merge operation is assisted by the write logs, it is referred to as a minimerge. The copy assist reduces system resource usage and copy times by enabling a direct disk-to-disk transfer of data without going through host node memory.
Assisted merge operations are usually too short to be noticeable. Improved performance is also possible during the assisted copy operation because it consumes less CPU and interconnect resources. Although the primary purpose of the performance assists is to reduce the system resources required to perform a copy or merge operation, in some circumstances you may also observe improved read and write I/O performance.
Volume Shadowing for OpenVMS supports both assisted and unassisted shadow sets in the same OpenVMS Cluster configuration. Whenever you create a shadow set, add members to an existing shadow set, or boot a system, the shadowing software reevaluates each device in the changed configuration to determine whether it is capable of supporting either the copy assist or the minimerge. Enhanced performance is possible only as long as all shadow set members are configured on controllers that support performance assist capabilities. If any shadow set member is connected to a controller without these capabilities, the shadowing software disables the performance assist for the shadow set.
When the correct revision levels of software are installed, the copy assist and minimerge are enabled by default, and are fully managed by the shadowing software.
The copy assist and minimerge are designed to reduce the time needed to do copy and merge operations. In fact, you may notice significant time reductions on systems that have little or no user I/O occurring during the assisted copy or merge operation. Data availability is also improved because copy operations quickly make data consistent across the shadow set.
Minimerge Performance Improvements
The minimerge feature provides a significant reduction in the time needed to perform merge operations. By using controller-based write logs, it is possible to avoid the total volume scan required by earlier merge algorithms and to merge only those areas of the shadow set where write activity was known to be in progress at the time the node or nodes failed.
Unassisted merge operations often take several hours, depending on user I/O rates. Minimerge operations typically complete in a few minutes and are usually undetectable by users.
The exact time taken to complete a minimerge depends on the amount of outstanding write activity to the shadow set when the merge process is initiated, and on the number of shadow set members undergoing a minimerge simultaneously. Even under the heaviest write activity, a minimerge completes within several minutes. Additionally, minimerge operations consume minimal compute and I/O bandwidth.
Copy Assist Performance Improvements
Copy times vary according to each configuration and generally take longer on systems supporting user I/O. Performance benefits are achieved when the source and target disks are on different HSJ internal buses.