HP OpenVMS Systems Documentation

Order Number: AA--Q2HLE--TE

This guide describes the security features available through the OpenVMS operating system. It explains the purpose and proper application of each feature in the context of specific security needs.

Revision/Update Information: This manual supersedes the OpenVMS Guide to System Security, Version 7.2

Software Version: OpenVMS Alpha Version 7.3
OpenVMS VAX Version 7.3

Compaq Computer Corporation Houston, Texas

© 2001 Compaq Computer Corporation

Compaq, VAX, VMS, and the Compaq logo Registered in U.S. Patent and Trademark Office.

OpenVMS and Tru64 are trademarks of Compaq Information Technologies Group, L.P. in the United States and other countries.

Microsoft, MS-DOS, Visual C++, Windows, and Windows NT are trademarks of Microsoft Corporation in the United States and other countries.

All other product names mentioned herein may be trademarks of their respective companies.

Confidential computer software. Valid license from Compaq required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Compaq shall not be liable for technical or editorial errors or omissions contained herein.

The information in this document is provided "as is" without warranty of any kind and is subject to change without notice. The warranties for Compaq products are set forth in the express limited warranty statements accompanying such products. Nothing herein should be construed as constituting an additional warranty.

ZK6346

The Compaq OpenVMS documentation set is available on CD-ROM.

Intended Audience

This guide is designed for users and for administrators responsible for protecting operating systems from tampering, observation, or theft of services by unauthorized users. The term security administrator is used in this guide to refer to the person or persons responsible for system security.

Document Structure

This guide contains the following information:

• Part 1: Overview
  Gives security administrators an overview of security issues, conceptual design features, and security features specific to OpenVMS systems.
  • Chapter 1 discusses levels of security requirements and describes three sources of security failures.
  • Chapter 2 introduces the reference monitor concept of security design and provides an overview of the operating system's security features.
• Part 2: Security for the User
  Describes security actions and features for the general user.
  • Chapter 3 provides information for the general user about the login and logout processes and the responsible use of passwords.
  • Chapter 4 and Chapter 5 describe object protection features in detail.
• Part 3: Security for the System Administrator
  Describes security actions and features for the security administrator.
  • Chapter 6 describes the general tasks of a security administrator.
  • Chapter 7 describes methods of controlling system access.
  • Chapter 8 describes methods of controlling access to system data and resources.
  • Chapter 9 describes security-auditing features.
  • Chapter 10 describes how to recognize when a system is under attack and how to protect and defend your system.
  • Chapter 11 describes security-related actions specific to clustered systems, such as setting up common system files and synchronizing authorization data.
  • Chapter 12 describes security considerations for systems using networking.
  • Chapter 13 describes how to set up and manage protected subsystems.
  • Appendix A provides a summary of all the user privileges available on the operating system and describes who may need them.
  • Appendix B lists the protection codes and ownership that Compaq provides for critical system files.
  • Appendix C describes how to operate OpenVMS systems in a Division C, Class 2 (C2) security environment.
  • Appendix D provides examples of security alarm messages.
• The Glossary provides definitions of security-related terms introduced in this guide.

Related Documents

The OpenVMS Guide to System Security assumes you are familiar with the reference material in the OpenVMS System Management Utilities Reference Manual pertaining to the following security-related utilities:

• Access control list editor (ACL editor)
• Accounting utility
• Audit Analysis utility
• Authorize utility
• Backup utility
• System Management (SYSMAN) utility

You might find helpful the amplified security information in the following manuals:

• OpenVMS DCL Dictionary
• OpenVMS System Manager's Manual
• OpenVMS Cluster Systems

For additional information about Compaq OpenVMS products and services, access the Compaq website at the following location:

http://www.openvms.compaq.com/

Reader's Comments

Compaq welcomes your comments on this manual. Please send comments to either of the following addresses:

How to Order Additional Documentation

Use the following World Wide Web address to order additional documentation:

http://www.openvms.compaq.com/

If you need help deciding which documentation best meets your needs, call 800-282-6672.

Conventions

The following conventions are used in this manual:

• Sources of security failures ( Section 1.1)
• Levels of security requirements ( Section 1.2)
• Reference monitor concept of security design ( Section 2.1)
• Security features of the operating system ( Section 2.2)

Effective operating system security measures help prevent unauthorized access and theft of computer time and any kind of sensitive information, such as marketing plans, formulas, or proprietary software. These measures can also protect equipment, software, and files from damage caused by tampering.

This chapter provides security administrators with an overview of security measures available with the operating system. Part 3 provides more specific information regarding site security policies, the tasks of security administrators, and methods of protecting site computer systems and resources.

1.1 Types of Computer Security Problems

On any system there can be two types of users: authorized and unauthorized. Any person authorized to use the computer system has the right to access the system and its resources according to the authorization criteria set up by the site security administrator. Usage criteria may include the time of day, types of logins, use of different resources like printers and terminals, and so on. Unauthorized users have no right to use the system at all or only at a given time of day, or they have no right to use certain system resources.

On a computer system, security breaches usually result from one of four types of actions:

• User irresponsibility refers to situations where the user purposely or accidentally causes some noticeable damage. One example would be a user who is authorized to access certain files making a copy of a key file to sell.
  There is little that an operating system can do to protect sites from this source of security failure. The problem frequently lies in application design deficiencies or inconsistent use of available controls by users and the security administrator. Sometimes the failure to enforce adequate environmental security unwittingly encourages this type of security problem.
  Even the best security system will fail if implemented inconsistently. This, along with the failure to motivate your users to observe good security practices, will make your system vulnerable to security failures caused by user irresponsibility. Chapter 3 discusses what users can do to help maintain system security.
• User probing refers to situations where a user exploits insufficiently protected parts of the system. Some users consider gaining access to a forbidden system area as an intellectual challenge, playing a game of user versus system. Although intentions may be harmless, theft of services is a crime. Users with more serious intent may seek confidential information, attempt embezzlement, or even destroy data by probing. Always treat user probing seriously.
  The system provides many security features to combat user probing. Based on security needs, the security administrator implements features on either a temporary or permanent basis. See Chapter 4 for information on protecting data and resources with protection codes and access control lists.
• User penetration refers to situations where the user breaks through security controls to gain access to the system. While the system has security features that make penetration extremely difficult, it is impossible to make any operating system completely impenetrable.
  A user who succeeds in penetrating a system is both skilled and malicious. Thus, penetration is the most serious and potentially dangerous type of security breach. With proper implementation of the OpenVMS security features, however, it is also the rarest security breach, requiring unusual skills and perseverance.
• Social engineering refers to situations in which an intruder gains access to a system not by technical means, but by deceiving users, operators, or administrators. Potential intruders may impersonate authorized users over the phone. Potential intruders may request information that gains them access to the system, such as telephone numbers or passwords, or they may request an unwitting operator to perform some action that compromises the security of the system.
  As the technical security features of operating systems have strengthened in recent years, social engineering has been a factor in a growing percentage of security incidents. Operator training, administrative procedures, and user awareness are all critical factors to ensure that access is not inadvertantly granted to unauthorized persons.

The following chapters explain how to avoid these problems:

• Chapter 8 explains how to augment the protection of system files and resources.
• Chapter 7 describes the intrusion detection system and how to set its parameters.
• Chapter 9 explains how to monitor system activity and be notified by malicious activity.
• Chapter 10 suggests how to handle system intrusions.
• Chapter 3 and Chapter 6 list topics to include in your site training programs.

Each site has unique security requirements. Some sites require only limited measures because they are able to tolerate some forms of unauthorized access with little adverse effect. At the other extreme are those sites that cannot tolerate even the slightest probing, such as strategic military defense centers. In between are many commercial sites, such as banks.

While there are many considerations in determining your security needs, the questions in Table 1-1 can get you started. Your answers can help determine the levels of your security needs. Also refer to Section 6.2 for a more specific example of site security requirements.

If you can tolerate most of the events listed, your security requirements are quite low. If your answers are mixed, your requirements are in the medium to high range. Generally, those sites that are most intolerant to the listed events have very high levels of security requirements.

When you review your site's security needs, do not confuse a weakness in site operations or recovery procedures as a security problem. Ensure that your operations policies are effective and consistent before evaluating your system security requirements.

1.3 Building a Secure System Environment

There are two sources of security problems outside the operating system domain: employee carelessness and facility vulnerability. If you have a careless or malicious employee or your facility is insecure, none of the security measures discussed in this guide will protect you from security breaches.

Most system penetration occurs through these environmental weaknesses. It is much easier to physically remove a small reel of tape than it is to break access protection codes or change file protection.

Compaq strongly encourages you to stress environmental considerations as well as operating system protection when reviewing site security.

This book discusses operating system security measures. When deciding which of these measures to implement, it is important for you to assess site security needs realistically. While instituting adequate security for your site is essential, instituting more security than actually necessary is costly and time-consuming.

When deciding which security measures to apply to your system, remember the following:

• The most secure system is also the most difficult to use.
• Increasing security can increase costs in terms of slower access to data, slower machine operations, and slower system performance.
• More security measures require more personnel time.

The operating system provides the basic mechanisms to control access to the system and its data. It also provides monitoring tools to ensure that access is restricted to authorized users. However, many computer crimes are committed by authorized users with no violation of the operating system's security controls.

Therefore, the security of your operation depends on how you apply these security features and how you control your employees and your site. By first building appropriate supervisory controls into your application and designing your application with the goal of minimizing opportunities for abuse, you can then implement operating system and site security features and produce a less vulnerable environment. For an example of one organization's security plan, see Chapter 6.

If you require your system to meet the United States government rating of a C2 secure operating system, please refer to Appendix C in this manual.

If you need a higher level of computer security for your OpenVMS secure system, Compaq offers SEVMS, which is the security enhanced version of OpenVMS that provides mandatory access controls to enforce a system-wide security policy.

SEVMS is a U.S. Department of Defense B1-rated secure operating system.