HP OpenVMS Systems Documentation

OpenVMS Alpha Partitioning and Galaxy Guide

Order Number: AA--REZQC--TE

April 2001

This guide describes how to use hard partitions, soft partitions (OpenVMS Galaxy), and resource affinity domains (RADs) with OpenVMS Alpha on AlphaServer systems that support these features. In addition, it describes how to create, manage, and use an OpenVMS Galaxy computing environment.

Revision/Update Information: This supersedes the OpenVMS Alpha Version 7.2--1 manual.

Software Version: OpenVMS Alpha Version 7.3

Compaq Computer Corporation
Houston, Texas

Compaq, DECnet, VAX, VMS, and the Compaq logo Registered in the U.S. Patent and Trademark Office.

OpenVMS is a trademark of Compaq Information Technologies Group, L.P. in the United States and other countries.

Motif, OSF/1, and UNIX are trademarks of The Open Group in the United States and other countries.

All other product names mentioned herein may be trademarks of their respective companies.

Confidential computer software. Valid license from Compaq required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

The information in this document is provided "as is" without warranty of any kind and is subject to change without notice. The warranties for Compaq products are set forth in the express limited warranty statements accompanying such products. Nothing herein should be construed as constituting an additional warranty.

ZK6512

The Compaq OpenVMS documentation set is available on CD-ROM.

Contents

Index

Preface

The OpenVMS Alpha Partitioning and Galaxy Guide describes how customers can take advantage of the partitioning and OpenVMS Galaxy capabilities available in OpenVMS Alpha Version 7.3.

The information in this document applies to OpenVMS Alpha systems only; it does not apply to OpenVMS VAX systems.

Intended Audience

This guide is intended for system managers, application programmers, technical consultants, data center managers, and anyone else who wants to learn about OpenVMS Galaxy and the partitioning capabilities of OpenVMS Alpha.

Document Structure

The OpenVMS Alpha Partitioning and Galaxy Guide introduces OpenVMS partitioning concepts and features on the hardware platforms that support them. It also explains how to use the OpenVMS Galaxy capabilities available in OpenVMS Alpha Version 7.3.

This guide covers the following OpenVMS Galaxy and partitioning topics:

Basic concepts
Hardware and software requirements
Configuration alternatives
Licensing information
Installation procedures
System management options
Descriptions of the application programming interfaces (APIs) available to develop programs that take advantage of OpenVMS Galaxy features

The OpenVMS Alpha Partitioning and Galaxy Guide assumes that readers are familiar with OpenVMS concepts and operation, and it does not cover basic OpenVMS information.

Reader's Comments

Compaq welcomes your comments on this manual.

Print or edit the online form SYS$HELP:OPENVMSDOC_COMMENTS.TXT and send us your comments by:

Internet	openvmsdoc@compaq.com
Mail	Compaq Computer Corporation OSSG Documentation Group, ZKO3-4/U08 110 Spit Brook Rd. Nashua, NH 03062-2698

How To Order Additional Documentation

Use the following World Wide Web address for information about how to order additional documentation:

http://www.openvms.compaq.com/

If you need help deciding which documentation best meets your needs, call 1-800-282-6672.

Conventions

In this manual:

Instance refers to a copy of the OpenVMS Alpha operating system.
OpenVMS Galaxy and Galaxy refer to the Compaq Galaxy Software Architecture on OpenVMS.
DECwindows and DECwindows Motif refer to DECwindows Motif for OpenVMS software.

The following conventions are used in this manual:

Ctrl/ x	A sequence such as Ctrl/ x indicates that you must hold down the key labeled Ctrl while you press another key or a pointing device button.
PF1 x	A sequence such as PF1 x indicates that you must first press and release the key labeled PF1 and then press and release another key or a pointing device button.
`[Return]`	In examples, a key name enclosed in a box indicates that you press a key on the keyboard. (In text, a key name is not enclosed in a box.) In the HTML version of this document, this convention appears as brackets, rather than a box.
...	A horizontal ellipsis in examples indicates one of the following possibilities: Additional optional arguments in a statement have been omitted. The preceding item or items can be repeated one or more times. Additional parameters, values, or other information can be entered.
. . .	A vertical ellipsis indicates the omission of items from a code example or command format; the items are omitted because they are not important to the topic being discussed.
( )	In command format descriptions, parentheses indicate that you must enclose the options in parentheses if you choose more than one.
[ ]	In command format descriptions, brackets indicate optional elements. You can choose one, none, or all of the options. (Brackets are not optional, however, in the syntax of a directory name in an OpenVMS file specification or in the syntax of a substring specification in an assignment statement.)
\|	In command format descriptions, vertical bars separating items inside brackets indicate that you choose one, none, or more than one of the options.
{ }	In command format descriptions, braces indicate required elements; you must choose one of the options listed.
bold text	This text style represents the introduction of a new term or the name of an argument, an attribute, or a reason.
italic text	Italic text indicates important information, complete titles of manuals, or variables. Variables include information that varies in system output (Internal error number), in command lines (/PRODUCER= name), and in command parameters in text (where dd represents the predefined code for the device type).
UPPERCASE TEXT	Uppercase text indicates a command, the name of a routine, the name of a file, or the abbreviation for a system privilege.
`Monospace text`	Monospace type indicates code examples and interactive screen displays. In the C programming language, monospace type in text identifies the following elements: keywords, the names of independently compiled external functions and files, syntax summaries, and references to variables or identifiers introduced in an example.
-	A hyphen at the end of a command format description, command line, or code line indicates that the command or statement continues on the following line.
numbers	All numbers in text are assumed to be decimal unless otherwise noted. Nondecimal radixes---binary, octal, or hexadecimal---are explicitly indicated.

Chapter 1
Managing Workloads With Partitions and Resource Managemement

OpenVMS customers use systems that support hard and soft partitions in many different ways. To most effectively use these systems, customers can decide which configuration options best meet their computing and application needs.

This chapter describes how to use hard and soft partitions and the new OpenVMS support for resource affinity domains (RADs) to ensure that applications run as efficiently as possible on the new AlphaServer systems.

1.1 Using Hard and Soft Partitions on OpenVMS Systems

Hard partitioning is a physical separation of computing resources by hardware-enforced access barriers. It is impossible to read or write across a hard partition boundary. There is no resource sharing between hard partitions.

Soft partitioning is a separation of computing resources by software-controlled access barriers. Read and write access across a soft partition boundary is controlled by the operating system. OpenVMS Galaxy is an implementation of soft partitioning.

The way that customers choose to partition their new AlphaServer GS series systems depends on their computing environments and application requirements. When deciding how to configure an OpenVMS system that supports partitioning, customers need to consider the following questions:

How many hard partitions do I need?
How many soft partitions do I need?
How small can I make the partitions?

1.2 OpenVMS Partitioning Guidelines

When deciding whether to use hard or soft partitons on the new AlphaServer GS series systems, you should be aware of the following:

Hard partitions must be on Quad Building Block (QBB) boundaries. (Note that a hard partition can contain more than one QBB.)
Soft partitions (Galaxy instances) do not have to be on QBB boundaries, but your configuration will be easier to understand and maintain if they are.
Each partition (hard or soft) must have its own console line. Note that each QBB in the system can have one console line in it. Therefore, the maximum number of partitions (hard or soft) in the system equals the number of QBBs in the system.
You can have multiple soft partitions within a hard partition.
You only need one cluster license for the entire AlphaServer GS series system. It does not matter how many instances exist or how they are clustered, internally or externally. (Note that this is different from the AlphaServer 8400 system.)

1.3 Creating Hard Partitions

Each hard partition requires the following:

Standard COM1 UART console line for each partition
PCI drawer for each partition
An I/O module per partition
At least one CPU module per partition
At least one memory module per partition
Memory options should be selected in the context of an application's sensitivity to memory bandwidth and memory capacity, and the number of hardware partitions. This will determine the number of memory base modules and upgrades needed. The total capacity required will determine the size of the arrays to be chosen.
Memory modules should be configured in powers of 2: That is, 0, 1, 2, or 4 base modules in a QBB. Upgrades should also be installed in powers of 2: 0, 1, 2, or 4 base modules in a QBB.

The following sections describe three hard partition configuration examples:

Example configuration 1 has four QBBs and four hard partitions. Each hard partition contains one QBB.
Example configuration 2 has four QBBs and two hard partitions. Each hard partition contains two QBBs.
Example configuration 3 has four QBBs and two hard partitions. One partition contains one QBB, and the other contains three QBBs.

For more information about the partitioning procedures described in this section, see the AlphaServer GS80/160/320 Firmware Reference Manual.

1.3.1 Hard Partition Configuration Example 1

This example configuration uses four QBBs to configure four hard partitions. Each hard partition contains one QBB.

Configuration Example 1
   -------------------
  | NODE   | HP | QBB |
  | -------|----|-----|
  | WILD7  |  0 |   0 |
  | WILD8  |  1 |   1 |
  | WILD9  |  2 |   2 |
  | WILD10 |  3 |   3 |
   -------------------

To configure an AlphaServer GS160 system with four hard partitions, perform the following sequence of SCM commands:

From the SCM console enter the following settings for the hp NVRAM variables. Note that the values are bit masks.


SCM_E0> power off -all

SCM_E0> set HP_count 4
SCM_E0> set HP_qbb_mask0 1
SCM_E0> set HP_qbb_mask1 2
SCM_E0> set HP_qbb_mask2 4
SCM_E0> set HP_qbb_mask3 8
SCM_E0> set HP_qbb_mask4 0
SCM_E0> set HP_qbb_mask5 0
SCM_E0> set HP_qbb_mask6 0
SCM_E0> set HP_qbb_mask7 0

SCM_E0> power on -all

You can also power off/on individual hard partitions. For example, using this configuration, you would:

SCM_E0> power off -all
SCM_E0> set HP_count 4
SCM_E0> set HP_qbb_mask0 1
SCM_E0> set HP_qbb_mask1 2
SCM_E0> set HP_qbb_mask2 4
SCM_E0> set HP_qbb_mask3 8
SCM_E0> set HP_qbb_mask4 0
SCM_E0> set HP_qbb_mask5 0
SCM_E0> set HP_qbb_mask6 0
SCM_E0> set HP_qbb_mask7 0
SCM_E0> power on -partition 0
SCM_E0> power on -partition 1
SCM_E0> power on -partition 2
SCM_E0> power on -partition 3

During the powering up phases of each hard partition, status information is displayed showing how the partitions are coming online. Pay close attention to this information and confirm that there are no failures during this process.

As each hard partition comes online, you will be able to start working with that hard partition's console device. Also note, that depending on the setting of the NVRAM variable AUTO_QUIT_SCM, each hard partition's console will come on line in either the SCM or SRM console mode.

From each hard partition's console, you would then enter into the SRM console and configure any console variables specific to that hard partition. After that, you boot OpenVMS in each hard partition according to standard OpenVMS procedures. For example:

Hard partition 0 typical SRM console settings for OpenVMS:

P00>>>show bootdef_dev
bootdef_dev             dkb0.0.0.3.0
P00>>>show boot_osflags
boot_osflags            0,0
P00>>>show os_type
os_type                 OpenVMS

Hard Partition 1 typical SRM console settings for OpenVMS:

P00>>>show bootdef_dev
bootdef_dev             dkb0.0.0.3.0
P00>>>show boot_osflags
boot_osflags            1,0
P00>>>show os_type
os_type                 OpenVMS

Hard Partition 2 typical SRM console settings for OpenVMS:

P00>>>show bootdef_dev
bootdef_dev             dkb0.0.0.3.0
P00>>>show boot_osflags
boot_osflags            2,1
P00>>>show os_type
os_type                 OpenVMS

Hard Partition 3 typical SRM console settings for Compaq Tru64 UNIX:

P00>>>show bootdef_dev
bootdef_dev             dka0.0.0.1.16
P00>>>show boot_osflags
boot_osflags            A
P00>>>show os_type
os_type                 UNIX

1.3.2 Hard Partition Configuration Example 2

This example configuration uses four QBBs to configure two hard partitions. Each hard partition contains two QBBs.


Configuration Example 2
   --------------------
   |NODE   | HP | QBB |
   |-------|----|-----|
   |WILD7  |  0 |  0,1|
   |WILD8  |  1 |  2,3|
   --------------------

To configure an AlphaServer GS160 with two hard partitions, perform the following sequence of SCM commands:

From the SCM console enter the following settings for the hp NVRAM variables:

SCM_E0> power off -all

SCM_E0> set HP_count 2
SCM_E0> set HP_qbb_mask0 3
SCM_E0> set HP_qbb_mask1 c
SCM_E0> set HP_qbb_mask2 0
SCM_E0> set HP_qbb_mask3 0
SCM_E0> set HP_qbb_mask4 0
SCM_E0> set HP_qbb_mask5 0
SCM_E0> set HP_qbb_mask6 0
SCM_E0> set HP_qbb_mask7 0

SCM_E0> power on -all

As each hard partition comes on line, you'll be able to start working with that hard partition's console device.

From each hard partition's console, you would then, as in configuration example 1, enter into the SRM console, and configure any console variables specific to that hard partition, and then boot OpenVMS in each hard partition.

1.3.3 Hard Partition Configuration Example 3

Like Configuration 2, this configuration uses four QBBs to configure two hard partitions; the only difference is the changing of the number of QBBs per hard partition.

Configuration Example 3
    -------------------
   |NODE   | HP | QBB |
   |-------|----|-----|
   |WILD7  |  0 |0,1,2|
   |WILD8  |  1 |    3|
    -------------------

To configure an AlphaServer GS160 system, perform the following sequence of SCM commands:

From the SCM console enter the following settings for the hp NVRAM variables:

SCM_E0> power off -all

SCM_E0> set HP_count 2
SCM_E0> set HP_qbb_mask0 7
SCM_E0> set HP_qbb_mask1 8
SCM_E0> set HP_qbb_mask2 0
SCM_E0> set HP_qbb_mask3 0
SCM_E0> set HP_qbb_mask4 0
SCM_E0> set HP_qbb_mask5 0
SCM_E0> set HP_qbb_mask6 0
SCM_E0> set HP_qbb_mask7 0

SCM_E0> power on -all

As in the other examples, as each hard partition comes online, you will be able to start working with that hard partition's console device.

1.3.4 Updating Console Firmware on AlphaServer GS80/160/320 Systems

To update the SRM console firmware on a system that is hard partitioned, you must do it separately for each hard partition. There is no way to update all of the firmware on each partition at one time.

1.4 OpenVMS Galaxy Support

OpenVMS Galaxy is an implementation of soft partitioning.

For information about OpenVMS Galaxy concepts, see Chapter 2.

1.4.1 Using Galaxy in Hard Partitions

You can create multiple soft partitions within a single hard partition by using the standard Galaxy procedures as described in Chapter 9.

Note that the Galaxy ID is within the hard partition. That is, if you have two hard partitions and you run Galaxy in both, each Galaxy will have its own unique Galaxy ID. Keep this in mind when you use network management tools; they will identify two Galaxy environments in this case.

1.5 OpenVMS Application Support for Resource Affinity Domains (RADs)

The large amount of physical memory in the new AlphaServer GS series systems provides opportunities for extremely large databases to be completely in memory. The AlphaServer nonuniform memory access (NUMA) system architecture provides the bandwidth to efficiently access this large amount of memory. NUMA is an attribute of a system in which the access time to any given physical memory is not the same for all CPUs.

In OpenVMS Alpha Version 7.2--1H1, OpenVMS engineering added NUMA awareness to OpenVMS memory management and process scheduling. This capability (application support for resource affinity domains) ensures that applications running in a single instance of OpenVMS on multiple QBBs can execute as efficiently as possible in a NUMA environment.

The operating system treats the hardware as a set of resource affinity domains (RADs). A RAD is a set of hardware components (CPUs, memory, and I/O) with common access characteristics. On AlphaServer GS80/160/320 systems, a RAD corresponds to a Quad Building Block (QBB). A CPU references memory in the same RAD approximately three times faster than it references memory in another RAD. Therefore, it is important to keep the code being executed and the memory being referenced in the same RAD as much as possible while not giving some processes a consistently unfair advantage. Good location is the key to good performance, but it must be as fair as possible when fairness is important.

The OpenVMS scheduler and the memory management subsystem work together to achieve the best possible location by:

Assigning each process a preferred or "home" RAD.
Assigning process-private pages from the home RAD's memory.
Usually scheduling a process on a CPU in its home RAD.
Replicating operating system read-only code on each RAD.
Distributing global pages over RADs.
Striping reserved memory over RADs.

For more information about using the OpenVMS RAD application programming interfaces, see Chapter 3.