HP OpenVMS Systems Documentation

You can enable statistics gathering in one of three ways:

• Through the DCL interface using the SET FILE/STATISTICS command
• From a program through the Create service using a $XABITM macro
• Through the FDL interface by assigning the FILE secondary attribute FILE_MONITORING when creating the file

• OpenVMS RMS statistics have not been enabled.
• The file has not been opened or connected.
• A restriction listed previously in Section 3.6 has been ignored.

RMS attributes:     File statistics enabled

Note that RMS creates a global page-file section when you initially open a file marked for statistics gathering. Excessive use of statistics gathering might exhaust resources associated with global sections, and if RMS cannot create the global page-file section, the $OPEN service returns an error. See the OpenVMS System Services Reference Manual: A--GETUAI for information about system parameters associated with global sections.

See the OpenVMS DCL Dictionary for details about using the DCL interface to enable statistics gathering for a file. The OpenVMS Record Management Utilities Reference Manual provides details about enabling RMS statistics for a file through the FDL interface. Instructions for gathering RMS statistics through the program interface are provided in the OpenVMS Record Management Services Reference Manual.

3.6.2 Using RMS Statistics

This section provides an example of how you can use RMS statistics to improve file processing performance. In this example, the system manager suspects that an I/O bottleneck involving the file DATABASE.DAT is causing a system performance problem. To confirm the suspicion, the system manager enables statistics monitoring on the file. Note that, if the system manager does not have sole access to the file, the SET FILE command returns a file access conflict error message.

The system manager invokes the Monitor utility to obtain a periodic sampling of RMS statistics that describe the processing activity related to DATABASE.DAT. The statistics relating to the operations rate, the buffer caching rate, the data rate, and the locking rate displayed on the Monitor screens provide the system manager with information for making decisions about where to place the file on disk and how to select optimal tuning parameters for the file.

Using the DCL interface, the system manager enables statistics gathering with the following command:

$  SET FILE/STATISTICS DATABASE.DAT

Having enabled RMS statistics, the system manager runs the application and then uses the DCL interface to periodically display the statistics using the following command:

$  MONITOR RMS/FILE=DATABASE.DAT/ITEM=CACHING

The Monitor utility responds with information displays similar to the following:

                             VAX/VMS Monitor Utility
                              RMS CACHE STATISTICS
                                 on node MENASH
                              28-FEB-1994
 16:03:59
(Index)  MENASH$DUA0:[TOREP]DATABASE.DAT;1
Active Streams:  60                    CUR        AVE        MIN        MAX

    Local Cache Hit Percent          40.00      59.86      38.00      80.00

    Local Cache Attempt Rate         59.81      59.26      56.07      60.55

    Global Cache Hit Percent         87.54      81.38      57.43     100.00

    Global Cache Attempt Rate        23.36      14.88       7.47      23.85

    Global Buf Read I/O Rate          0.00       1.21       0.00       2.91

    Global Buf Write I/O Rate         0.00       0.00       0.00       0.00

    Local Buf Read I/O Rate          12.14       8.84       4.67      13.08

    Local Buf Write I/O Rate         29.90      29.63      28.97      29.90

If this display represents a period of activity that the system manager or application developer wants to optimize, then the following observations can be derived:

• The number of Active Streams connected to this file and the cache attempt rates verify that this is a very active file that could benefit from performance analysis.
• Global buffer utilization is very good as indicated by Global Cache Hit Percent. The relatively few global buffer read I/Os and complete absence of global buffer write I/Os might be due to the fact that index buckets tend to be placed in global buffers. Index buckets are often read but seldom written. Generally, global buffer I/O activity can be reduced by adding global buffers. However, excessive use of global buffers can increase the elapsed time for RMS operations as measured in the application. The Run-Time Library timer routines can be used to measure elapsed time for RMS operations. You can derive an optimal number of global buffers by varying the number and then evaluating the global buffer I/O rates against elapsed time per operation.
• Local buffer read and write I/O rates indicate that the application might be saturating the capacity of the disk (RA81). You can verify this by running the MONITOR DISK option clusterwide and examining the queue length for the disk involved. It might help to spread the file over more than one disk.
  There is a tendency for index file data buckets to be placed into local buffers even when a large global buffer cache is present. By varying the number of local buffers and re-examining these statistics, you can pick an optimal number of local buffers. As always, you should check that any buffering changes do not result in additional paging I/Os.

$  MONITOR MONITOR RMS/FILE=DATABASE.DAT/ITEM=ALL

For more information about using the SET FILE/STATISTICS command, see the OpenVMS DCL Dictionary. The OpenVMS System Manager's Manual provides detailed information about using the Monitor utility. For more information on performance management, see Guide to OpenVMS Performance Management and A Comparison of System Management on OpenVMS AXP and OpenVMS VAX.

3.7 Processing in an OpenVMS Cluster Environment

This section discusses designing file applications for an OpenVMS Cluster and the performance you can reasonably expect from the OpenVMS Cluster environment.

Processing in an OpenVMS Cluster environment offers many advantages:

• Performance---In general, the performance of each node in an OpenVMS Cluster is similar to that of a single-node system that has the same processing load, assuming the aggregate I/O per disk drive is reasonable.
• Availability---With the appropriate configuration, a node that leaves the OpenVMS Cluster does not stop the OpenVMS Cluster.
• Flexibility---You can process shared applications on more than one node.
• Accessibility---Shared resources are very easy to use in an OpenVMS Cluster. The synchronized access to the data provides data integrity with no redundancy.

For more information about OpenVMS Clusters, see OpenVMS Cluster Systems.

3.7.1 OpenVMS Cluster Shared Access

Shared access is one of the chief advantages of processing in an OpenVMS Cluster environment. Many applications that run on a single-node system can run on an OpenVMS Cluster with no changes.

However, applications that access shared files in an OpenVMS Cluster incur some additional overhead for the OpenVMS Cluster synchronization; the amount of additional overhead depends on the locking requirements of your application.

3.7.1.1 Locking Considerations

The distributed lock manager allows several users to share files concurrently in an organized manner. RMS uses the lock manager to control file access.

The lock-mastering node controls the record and bucket locking for a given file for users on every node of the OpenVMS Cluster. Initially, it is the first node from which the file is opened. However, another node may become the lock-mastering node when a node either joins or leaves the OpenVMS Cluster.

The lock-mastering node may also change every time the file is opened. When another process opens the file (provided that the file was closed), the node on which that process resides becomes the new lock-mastering node for that file.

Lock requests issued by processes on the lock-mastering node incur less cost than lock requests issued from other nodes. Conversely, the lock-mastering node has the additional work of processing lock requests for that file for all other nodes.

The lock-requesting node is any node in the OpenVMS Cluster other than the lock-mastering node for a given file.

RMS locks buckets and records during record operations only if the file is open for shared writing. Conversely, RMS does no locking during record operations if the file is open for shared read-only access or for exclusive access.

Lock requests for root locks (top-level or parent locks) in an OpenVMS Cluster may be slightly slower than on a single-node system. However, these locks are used when you open and close files, so the time for lock operations is only a fraction of the total time needed to open and close files.

There is no performance difference between a single-node system and an OpenVMS Cluster if the file sharing takes place on a single node of the OpenVMS Cluster. Only when sharing spans across the OpenVMS Cluster nodes does distributed locking occur.

As a result, the record locking itself may take a little longer, but because you have multiple CPUs in the OpenVMS Cluster, your application benefits from the added processing power.

Sharing files in an OpenVMS Cluster also requires enough memory for nonpaged pool to store additional lock data structures. This requirement, however, is dependent upon your processing load.

3.7.1.2 I/O Considerations

Sharing files in an OpenVMS Cluster environment also means sharing resources, such as disks and other pieces of I/O hardware. When applications on many nodes share data on one disk, OpenVMS Cluster performance may degrade due to excessive I/O operations.

3.7.2 Performance Recommendations

Four general recommendations about performance in an OpenVMS Cluster environment are described in the following list:

• Estimate the I/O needs of your application. In an OpenVMS Cluster, and particularly with a shared file, multiple nodes can generate many I/O requests to a single disk. The capacity of the disk to handle I/O traffic can affect OpenVMS Cluster performance if you allow your applications to become I/O bound. The Monitor utility is a good tool for estimating how many I/O requests your application generates. For more information about the Monitor utility, see the OpenVMS System Manager's Manual.
• Process files with exclusive access to obtain better performance than processing files with shared-write access. Opening files for unnecessary shared-write access incurs needless locking cost (even on a single node system).
• If possible, confine your application to a single CPU. If sufficient CPU resources and I/O capacity are available, your application performs faster than if it was spread over many nodes.
• Provide for sufficient memory because the space overhead for the lock database and other system software can be significant.

After you have designed your file, you need to create it. First you must specify the file characteristics you selected during the design phase. Then you need to create the actual file with those characteristics and to protect it (decide who has access to the file). Lastly, you need to put records in the file, or "populate" it.

This chapter describes the process of creating and populating files.

• Section 4.1 tells how to select and specify file-creation characteristics.
• Section 4.2 describes how to create a file.
• Section 4.3 describes how to create and access tagged files.
• Section 4.4 explains how to define file protection.
• Section 4.5 describes how to populate the file.
• Section 4.6 provides a summary of the options related to file creation.

You can specify the characteristics you need to create a file in two ways.

If you use VAX MACRO or BLISS--32, you can specify file characteristics by including OpenVMS RMS (hereafter referred to as RMS) control blocks in your application program.

If you use a high-level language, you can use the File Definition Language (FDL), a special-purpose language that is used to write specifications for data files. Of course, you also have the option of using FDL with VAX MACRO or BLISS--32.

The following sections describe how you can specify file-creation characteristics by using RMS control blocks or by creating FDL files.

4.1.1 Using RMS Control Blocks

You can establish characteristics for the file you create by using an RMS file access block (FAB) and extended attribute blocks (XABs). These control blocks allow you to take the defaults that RMS provides or to override the defaults and define the characteristics that suit your particular application.

4.1.1.1 File Access Block

The FAB is made up of fields that describe various file characteristics and contain the following file-related information:

• The addresses of the file name string and the default name string
• The file organization
• The record format
• Information about disk storage space

The FAB lets you use both the creation-time characteristics and the run-time characteristics of RMS. You must define one FAB for each file your program opens or creates.

For more information about the FAB, see the OpenVMS Record Management Services Reference Manual.

4.1.1.2 Extended Attribute Blocks

Extended attribute blocks (XABs) are optional control blocks that contain supplementary file-attribute information. The following is a partial list of XABs that can be used to provide supporting file information:

• Initial size and extent information (XABALL)
• File protection (XABPRO)
• Key definition (XABKEY)
• Date and time information (XABDAT)

Like FABs, XABs allow you to use both the creation-time characteristics and the run-time characteristics of RMS.

With XABs, you can define various file attributes beyond those specified in the associated FABs.

For more information about the extended attribute blocks, see the OpenVMS Record Management Services Reference Manual.

4.1.2 Using File Definition Language

FDL provides a way to create data files using special text files called FDL files. FDL files are written in a file definition language, which permits you to specify appropriate attributes and values for the file.

You create and modify FDL files using the Edit/FDL utility (EDIT/FDL). The Edit/FDL utility contains built-in design algorithms to help you optimize data file design. The Edit/FDL utility recognizes correct FDL syntax and informs you immediately of syntax errors. (You can use a text editor or the DCL command CREATE to create an FDL file, but you must then follow the validity rules listed in the OpenVMS Record Management Utilities Reference Manual.)

You can also use the Analyze/RMS_File utility to create FDL files from existing data files. FDL files created in this manner contain special analysis sections that you can use with the Edit/FDL utility to tune your data files.

You can use the Create/FDL utility and the Convert utility to create data files from the specifications in the FDL files. Note that the Convert utility processes relative files by sequentially reading records from the input file, then writing them to the output file. As a result, the relative record numbers (RRN) change when the input file contains deleted or unused records.

By using an FDL file to create a data file from a high-level language, you can specify most of the creation-time characteristics that are available with RMS control blocks (FABs and XABs). However, to use all of the connect-time features, including wildcard characters, you must use the RMS control blocks.

4.1.2.1 Using the Edit/FDL Utility

You can use the Edit/FDL utility in two ways: with a terminal dialog (interactively) or without one (noninteractively).

If you use the Edit/FDL utility noninteractively, you can execute only the OPTIMIZE script. The OPTIMIZE script lets you optimize an existing FDL file without an interactive session. For more information, see Section 10.3.

Alternatively, if you use the Edit/FDL utility interactively, you can use all the scripts, each of which has a series of menus. When you invoke the Edit/FDL utility, it displays a main menu. To select a menu item, you only need to enter the first letter of the item because each selection has a unique first letter.

Table 4-1 summarizes the Edit/FDL utility commands.

Ctrl/Z is equivalent to the EXIT command if you use it at the main menu level. If you use it from any other level, Ctrl/Z returns you to the main menu level.

In most cases, a command from the main menu brings up a second level menu. For instance, typing the ADD command displays the following menu:

                    Legal Primary Attributes

ACCESS  attributes set the run-time access mode of the file
AREA x  attributes define the characteristics of file area x
CONNECT attributes set various RMS run-time options
DATE    attributes set the date parameters of the file
FILE    attributes affect the entire RMS data file
KEY y   attributes define the characteristics of key y
NETWORK attributes set run-time network access parameters
RECORD  attributes set the non-key aspects of each record
SHARING attributes set the run-time sharing mode of the file
SYSTEM  attributes document operating system-specific items
TITLE     is the header line for the FDL file

Enter desired primary     (Keyword)[FILE] :

One of the most important features of the Edit/FDL utility is that it helps you create FDL files that define indexed, relative, and sequential data files. To do this, the Edit/FDL utility provides seven scripts that guide you through an interactive session. You can choose one of these scripts at the start of a session, or you can instruct the Edit/FDL utility to automatically invoke a particular script each time that you enter the EDIT/FDL command.

Table 4-2 lists the seven scripts.

An interactive session is controlled by these Edit/FDL utility scripts. You can invoke a script in two ways:

• You can select the INVOKE command from the main menu and then choose your script. When you answer the script questions, the Edit/FDL utility displays a list of FDL attributes and their assigned values. At this point, you can use the Edit/FDL utility commands to further modify the attribute values or to end the editing session.
• You can begin a script by entering a DCL command in the following form:

  EDIT/FDL/SCRIPT=script-name

  This command bypasses the main menu to directly display the menu for the selected script.

Example 4-1 shows a sample session with the Edit/FDL utility.

                  OpenVMS FDL Editor

  Add     to insert one or more lines into the FDL definition
  Delete  to delete one or more lines from the FDL definition
  Exit    to leave the FDL Editor after creating the FDL file
  Help    to obtain information about the FDL Editor
(1)  Invoke  to initiate a script of related questions
  Modify  to change existing line(s) in the FDL definition
  Quit    to abort the FDL Editor with no FDL file creation
  Set     to specify FDL Editor characteristics
  View    to display the current FDL Definition
(2)  Main Editor Function            (Keyword)[Help] : INVOKE


                Script Title Selection

  Add_Key       modeling and addition of a new index's parameters
  Delete_Key    removal of the highest index's parameters
  Indexed       modeling of parameters for an entire Indexed file
(3)  Optimize      tuning of all indexes' parameters using file statistics
  Relative      selection of parameters for a Relative file
  Sequential    selection of parameters for a Sequential file
  Touchup       remodeling of parameters for a particular index
(4)  Editing Script Title            (Keyword)[-] : INDEXED
(5)  Target disk volume Cluster Size (1-1Giga)[3]    : 3
(6)  Number of Keys to Define        (1-255)[1]      : 1

  Line    Bucket Size vs Index Depth      as a 2 dimensional plot
  Fill    Bucket Size vs     Load Fill Percent     vs Index Depth
(7)  Key     Bucket Size vs         Key Length        vs Index Depth
  Record  Bucket Size vs        Record Size        vs Index Depth
  Init    Bucket Size vs Initial Load Record Count vs Index Depth
  Add     Bucket Size vs  Additional Record Count  vs Index Depth
(8)  Graph type to display           (Keyword)[Line] : LINE
(9)  Number of Records that will be Initially Loaded
  into the File                   (0-1Giga)[-]  : 100000
(10)  (Fast_Convert NoFast_Convert RMS_Puts)
  Initial File Load Method        (Keyword)[Fast] : FAST
(11)  Number of Additional Records to be Added After
  the Initial File Load           (0-1Giga)[0]    : 0






(12)  Key  0 Load Fill Percent        (50-100)[100]   : 100
(13)  (Fixed Variable)
  Record Format                   (Keyword)[Var]  : VARIABLE
(14)  Mean Record Size                (1-32229)[-]    : 80
(15)  Maximum Record Size             (0,80-32229)[0] : 0






(16)  (Bin2 Bin4  Bin8  Int2  Int4  Int8  Decimal  String  Collated
      Dbin2 Dbin4 Dbin8 Dint2 Dint4 Dint8 Ddecimal Dstring Dcollated)
  Key  0 Data Type                (Keyword)[Str]  : STRING
(17)  Key  0 Segmentation desired     (Yes/No)[No]    : NO
(18)  Key  0 Length                   (1-255)[-]      : 9
(19)  Key  0 Position                 (0-32220)[0]    : 0
(20)  Key  0 Duplicates allowed       (Yes/No)[No]    : NO
(21)  File Prolog Version             (0-3)[3]        : 3
(22)  Data Key Compression desired    (Yes/No)[Yes]   : YES
(23)  Data Record Compression desired (Yes/No)[Yes]   : YES
(24)  Index Compression desired       (Yes/No)[Yes]   : YES

        *|
        9|
        8|
  Index 7|
        6|
  Depth 5|
        4|
        3|  3 3
        2|      2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
        1|                                                  1 1 1 1 1 1 1 1
         +- + - - - + - - - - + - - - - + - - - - + - - - - + - - - - + - +
(25)            1       5        10        15        20        25        30  32
                          Bucket Size (number of blocks)

   PV-Prolog Version       3 KT-Key  0 Type     String EM-Emphasis  Flatter ( 3)
   DK-Dup Key  0 Values   No KL-Key  0 Length        9 KP-Key  0 Position      0
   RC-Data Record Comp    0% KC-Data Key Comp       0% IC-Index Record Comp   0%
   BF-Bucket Fill       100% RF-Record Format Variable RS-Mean Record Size    80
   LM-Load Method  Fast_Conv IL-Initial Load    100000 AR-Added Records        0
          (Type "FD" to Finish Design)
(26)          Which File Parameter (Mnemonic)[refresh]     : FD
(27)  Text for FDL Title Section      (1-126 chars)[null]
  : FDL_SESSION_EXAMPLE
(28)  Data File file-spec             (1-126 chars)[null]
  : EXAMPLE.DAT
(29)  (Carriage_Return Fortran None Print)
  Carriage Control                (Keyword)[Carr] : CARRIAGE_RETURN

  Emphasis Used In Defining Default:      (    Flatter_files   )
  Suggested Bucket Sizes:                 (      3     3    27 )
(30)  Number of Levels in Index:              (      2     2     1 )
  Number of Buckets in Index:             (     72    72     1 )
  Pages Required to Cache Index:          (    216   216    27 )
  Processing Used to Search Index:        (    168   168   766 )
(31)  Key  0 Bucket Size              (1-63)[3]       : 3
(32)  Key  0 Name                     (1-32 chars)[null]
  : SSNUM
(33)  Global Buffers desired          (Yes/No)[No]    : NO
(34)  The Depth of Key  0 is Estimated to be No Greater
  than 2 Index levels, which is 3 Total levels.
(35)  Press RETURN to continue (^Z for Main Menu)

                    OpenVMS FDL Editor

  Add     to insert one or more lines into the FDL definition
  Delete  to delete one or more lines from the FDL definition
  Exit    to leave the FDL Editor after creating the FDL file
(36)  Help    to obtain information about the FDL Editor
  Invoke  to initiate a script of related questions
  Modify  to change existing line(s) in the FDL definition
  Quit    to abort the FDL Editor with no FDL file creation
  Set     to specify FDL Editor characteristics
  View    to display the current FDL Definition
(37)  Main Editor Function            (Keyword)[Help] : EXIT
(38)  DISK$:[FOX.RMS]FDL_SESSION_EXAMPLE.FDL;1  40 lines