HP OpenVMS Systems

XFC introduction

The OpenVMS operating system has two types of caches -- the data cache and the metadata cache. The data cache is referred to as eXtended File Cache (XFC) and the metadata cache is called the eXtended Qio Processor (XQP). XFC caches contents of all files present on the disk, excluding directory files. XQP caches file headers and directory files.

Statistics provided by XFC

XFC provides various statistics of the system, which can be used for the overall system performance tuning. XFC provides the statistics at the system, volume, file and I/O level. The DCL commands or the XFC SDA extension can be used to get the XFC statistics. In most scenarios, DCL as well as XFC SDA extensions provide a similar type of statistics. The XFC SDA extension generally provides little more information than the corresponding DCL commands.

Some of the important statistic include Read hit rate, XFC cache memory, I/O size distribution, and so on. This paper explores about the various statistics that are provided by XFC.

HotFile statistics

Hotfile is a file in the system that records many activities in terms of read and write operations. In many environments it is observed that 80% of the I/O's that are issued are on 20% of the files present in the system. This being the case, it is very important to know which among the many files that are present in the system fall into this 20% category.

Hotfile – TOPQIO

On a given volume, the TOPQIO keyword instructs XFC to display the first ‘N’ entries in descending order of the number of IOs done to the file.

DCL command

Sample output

Interpreting the output

On the disk BRAMHA$DKB200, the maximum number of IOs are done to the file PERFORM_TEST.COM (389451, 1057, 0).

• The total number of IOs done to the file is 1177. Among these IOs, 1176 are read IOs and 1 write IO.
• The read hits of the file are 100%. This is excellent because all attempts to read the file were satisfied from the cache itself.
• Allocated pages of the file is 1 Page. This is a small temporary file and hence consumes a very small portion of XFC cache memory.

Hotfile – TOPHITRATE

On a given volume, the TOPHITRATE keyword instructs XFC to display the first ‘N’ entries in descending order of read hit rates of the file.

DCL command

Sample output

Interpreting the output

On the disk BRAMHA$DKB200, the file with maximum read hit rate is TRACE_SETUP.COM (389474, 1, 0).

• The total number of IOs done to the file is 5. Among these IOs, 4 are read IOs and 1 write IO.
• The read hits of the file are 100%. This is excellent because all attempts to read the file were satisfied from the cache itself.
• Allocated page of the file is one page. This is a small temporary file and hence consumes a very small portion of the XFC cache memory.

Memory statistics

The "SHOW MEM/CACHE" DCL command provides the XFC memory utilization statistics at the system level. It also provides other important statistics, such as read hit rate, and number of read/write IOs at the system level.

DCL command

Sample output

Interpreting the output

• XFC memory can grow to a maximum of 2GB, that is 2047.81 MB
• Currently XFC has allocated itself a memory of 80.9 MB
• The free memory with XFC is 0MB. Any subsequent request to cache a file would cause XFC to expand in size.
• The overall read hit rate of the system is 90%. This is a good hit rate.
• Out of the total IOs issued in the system, 77% of the IOs are read IOs.

SDA command

Sample output

Interpreting the output

The output is interpreted similar to that of the corresponding “SHOW MEM/CACHE” DCL command.

System statistics

The "SHOW MEM/CACHE" DCL command or the “XFC SHOW SUMMARY” SDA command provides the XFC statistic at the system level. Some of the important statistics that can obtained are read hit rate and the number of read/write IOs.

DCL command

Sample output

Interpreting the system output

• The overall read hit rate of the system is 90%. This is a very good hit rate.
• Out of the total IOs issued in the system 77% of the IOs are read IOs.
• In this system, the number of open files cached by XFC is 398 and the number of closed files cached by XFC is 312.

SDA command

Sample output

Interpreting the system output

• 71090 virtual read IOs and 20770 virtual write IOs are issued.
• Read hit rate is 90.05%. This is a very good hit rate.
• The total number of files in  the cache is 711. Out of these, 400 files are open files and 311 files are closed files.

Periodic system activity statistics

The "SHOW MEM/CACHE" DCL command provides the XFC memory utilization statistic at the system level. It also provides other important statistics such as read hit rate and the number of read/write IOs at the system level. XFC records the activity of the system on a periodic basis, which is at a regular interval every 10 minutes. XFC records various information about the system, which would be useful to track the behavior of the system. Some of the statistics captured are:
Number of read/write IOs
Read hit rate
Open/closed files in cache
XFC cache memory

SDA command

Sample output

Interpreting the output

Between the time intervals of 29-MAY 00:35 and 29-MAY 00:45, the following data was gathered:

• Number of read IOs were 315, number of write IOs were 25
• Read hit rate was 84%
• The XFC cache memory increased from 65.9 MB to 66.9MB

Systemwide IO Size distribution statistic

XFC also keeps track of all the IOs issued in the system based on the IO size i.e., 1 block or 2 blocks and so on.

For IOs of particular size, XFC keeps track of statistics such as:

• Number of read operations
• Number of write operations
• Number of read hits

This statistic provides information on the read hit of IOs based upon the IO size.

DCL command

Sample output

Interpreting the system output

• 2384 write IOs were of size 1 block. 28160 IOs were of size 1 block and among these 27428 IOs were a read hit.
• 27 read IOs and 3 write IOs issued were of size in the range of 127-256 blocks. None of them was a read hit.
• 53 read IOs and no write IOs issued were of size greater than 255 blocks. None of them was a read hit.

SDA command

Sample output

SDA> XFC SHOW SUMMARY/STATS
---------------------------
...
I/O Statistics - Distributions (MAX_IO_SIZE: 127 )
--------------------------------------------------
Transfer Size     Reads       Hit Rate  Writes ...
 (blocks)
    1             28196          97%     2384 ...
    2              6962          91%       24 ...
    3               403          76%       34 ...
...
Reads between 128 and 255 blocks since last reset:      27
Read hits between 128 and 255 blocks since last reset:   0 (0%)
Reads larger than 256 blocks since last reset:          53
Read hits larger than 256 blocks since last reset:       0 (0%)
Writes between 128 and 255 blocks since last reset:      3
Writes larger than 256 blocks since last reset:          0

Interpreting the system output

• 28196 read IOs and 2384 write IOs were of size 1 block. The read hit rate for 1 block IOs is 97%.
• 27 read IOs and 3 write IOs issued were of size in the range of 127-256 block. None of them was a read hit.
• 53 read IOs and no write IOs that were issued were of size greater than 255 blocks. None of them was a read hit.

Volume statistics

The "SHOW MEM/CACHE=VOL=<VOL-NAME>" DCL command or the “SDA>XFC SHOW VOLUME/NAME=<VOL-NAME>” command provides the XFC statistics at the volume level. Some of the important statistics are read hit rate, and the number of read/write IOs.

DCL command

Sample output

Interpreting the output

On the volume BRAMHA$DKA100:

• The number of open files in cache is 398 and the number of closed files in the cache is 314.
• This volume has consumed 9715 pages of the total XFC cache memory
• The read hit rate of the volume is 92%. This is a very good read hit rate.
• Total IOs done on this volume is 85299. Out of this amount, 66304 IOs are virtual reads and 18995 IOs are virtual writes.

SDA command

Sample output

Interpreting the output

On the DISK$BR_V84R volume:

• The number of open files in cache is 400 and the number of closed files in the cache is 312.
• The hit rate of the volume is 71%.
  Note: This is its hit rate and NOT read hit rate, where
  Hit rate = Read hits / (virtual reads + virtual writes)
• Total IOs done on this volume is 85447. Out of this amount, 66452 IOs are virtual reads and 18995 IOs are virtual writes.
• XFC computes the IO response time for various IOs that are issued to this volume. I.e., on any given volume, for each IOs, XFC keeps track of the following:
  - IOs that start and end on the same CPU and are cache hits
  - IOs that start and end on the same CPU and are cache misses

File statistics

The "SHOW MEM/CACHE=FILE=<FILE-NAME>" DCL command or the “SDA>XFC SHOW FILE/ID=<FILD-IN-HEX>” command provides the XFC statistics at the file level. Some of the important statistics are read hit rate, number of read/write IOs, and XFC cache memory consumed by the file.

DCL command

Sample output

Interpreting the output

For the file PERFORM_TEST.COM (389451, 1057, 0),

• This file has consumed one page of the total XFC cache memory.
• The read hit rate of the file is 100%. This means that all attempts to access the file were satisfied from the cache itself.
• Total IOs done on this file is 1177. Out of these, 1176 IOs are virtual reads and 1 IO is virtual write.

SDA command

Sample output

Interpreting the output

For the file ERRLOG.SYS (9064, 1, 0):

• Total IOs done on this file is 2297. Out of these, 1530 IOs are virtual reads and 767 IOs are virtual writes.
• The hit rate of the file is 53.11%.
  (Note: This is the hit rate and NOT read hit rate).
  Hit Rate = read hits / (virtual reads + virtual writes)
• There has been 1 truncate, 765 access and 765 deaccess operations on the file
• XFC computes the IO response time for various IOs that are issued to this file.
  I.e. On the file, for each IOs, XFC keeps track of:
  - IOs that are cache hits which start and end on the same CPU
  - IOs that are cache miss which start and end on the same CPU

IO statistics

XFC creates a context block for every IO that it handles. The context block stores various information about the IO request such as IRP, operation start time, current IO phase and so on. At any given point of time, the context block indicates the current state of the IO.

IO statistics are generally used for debugging purpose. While analyzing the crash dump, you may be interested to know the current state of an IO request. In such cases, if the IO request has gone through XFC, then XFC would have created a context block for that IO request. The context block can then be examined to know the current state of the IO request.

If XFC is handling multiple IO requests, then it would have created multiple context blocks, one for each IO request. In such cases, you must examine the context block that corresponds to the IO request that you are looking for. This is done by examining the list of all the context blocks and then selecting a particular context block based on whether it's IRP field corresponds to the IRP we are looking for.

SDA command

Sample output

Interpreting the output

For the context block FFFFFFFF89379910:

XFC received the IO request on "31-MAY-2010 03:04:47.32".

• The IO request is for a write operation.
• The file id of the file to which this IO is done is "398F"
• This IO request is not a cache hit and hence the data has to be fetched from the disk
• Other structures associated with this IO are:
  • IRP - FFFFFFFF8916DBC0 --> IO request packet
  • PCB - FFFFFFFF896B0CC0 --> process control block

Summary

The techniques described here allow a user to get the XFC statistics at the system/volume/file/IO level. These statistics should then be analyzed to determine the hot files in the system at the volume level and also if the overall system performance is acceptable or not.

One of the important parameters that impacts the system performance is the read hit rate of the system /volume/file, which depends on a number of factors such as:

• Number of local read/write IOs to the file
• Number of clusterwide read/write IOs to the file

Once we collect the XFC statistics, for example:

• Read hit rate of the system is 50%

This means the overall cache read hit rates on the system is 50%. This is very low and hence there is a need to tune the system to increase the overall cache read hit rates. This would involve analyzing the IOs issued on the system, i.e. on different volumes by various applications.

• Read hit rate of the volume V1 is 50%

This means that the cache hit rate on the volume V1 is 50%. This is very low and hence there is a need to tune the system with respect to the IO operations on the volume to increase the read hit rate of the volume. This would involve analyzing the IOs issued on the volume V1 by various applications.