HP OpenVMS Systems Documentation

This command defines the THREADCP command verb:

   $ SET COMMAND SYS$UPDATE:THREADCP.CLD

This command displays the current settings of both thread control bits for the image TEST.EXE:

   $ THREADCP/SHOW TEST.EXE

This command displays the current settings of both thread control bits for all SYS$SYSTEM images:

   $ THREADCP/SHOW SYS$SYSTEM:*

This command sets both thread control bits explicitly for the image TEST.EXE:

   $ THREADCP/ENABLE=(MULTIPLE_KERNEL_THREADS, UPCALLS) TEST.EXE

This command clears both thread control bits explicitly for the image TEST.EXE:

   $ THREADCP/DISABLE=(MULTIPLE_KERNEL_THREADS, UPCALLS) TEST.EXE

On OpenVMS Alpha systems, a program can call the $GETJPI system service and specify the appropriate MULTITHREAD item code to determine whether kernel threads are in use. The return values have the same meanings as are defined for the MULTITHREAD system parameter, as summarized in Table B-5.

B.12.4 Creation of Virtual Processors

In general, there is no reason to create more virtual processors than there are physical processors; that is, the virtual processors would contend with each other for the physical processors and cause unnecessary overhead. Regardless of the value of the MULTITHREAD parameter, the Threads Library creates no more virtual processors than there are user threads (excluding internal threads).

The Threads Library does not delete virtual processors or let them terminate. They are retained in the HIB idle state until they are needed again. During image rundown, they are deleted by OpenVMS.

The Threads Library scheduler can schedule any user thread onto any virtual processor. Therefore, a user thread can run on different kernel threads at different times. Normally, this should pose no problem; however, for example, a user thread's PID (as retrieved by querying the system) can change from time to time.

B.12.5 Delivery of ASTs

When a user mode AST becomes deliverable to a Threads Library process, the OpenVMS scheduler makes an upcall to the Threads Library, passing the information that is required to deliver the AST (service routine address, argument, and target user thread ID). The Threads Library stores this information and queues the AST to be delivered to the appropriate user thread. That thread is made runnable (if it is not already), and executes the AST routine the next time it is scheduled to run. This means the following:

• A per-thread AST will interrupt the user thread that requested it, regardless of on which virtual processor the thread is running.
• The AST will be run at the priority of the target thread, so that low-priority threads' ASTs do not preempt or interfere with the execution of high-priority threads.
• The AST routine executes in the context of the target thread, so that the danger of surprise stack overflows is diminished, and stack-walks and exception propagation work as they should.

In addition to per-thread ASTs, there are also user mode ASTs that are directed either to the process as a whole, or to no thread in particular, or to a thread that has since terminated. These "process" ASTs are queued to the initial thread, making the thread runnable in a fashion similar to per-thread ASTs. They are executed in the context of the initial thread, for the following reasons:

• The initial thread has an expandable stack, unlike the other threads, which minimizes the danger of stack space problems.
• Any code that is making assumptions about specific characteristics of AST delivery is most likely running in the initial thread, so delivering the AST to the initial thread is least likely to cause problems.
• To ensure that the process ASTs are executed promptly, the initial thread gets a boost to the top scheduling priority. Because these ASTs cannot be associated with a particular thread, their priority cannot be assessed, so it is important that they be delivered promptly in the event that a high-priority thread is waiting to be signaled by one of them.

The following implications must be considered for application development:

• If an application makes heavy use of ASTs, it can starve the initial thread to a degree, because only that thread executes the ASTs that are directed to the entire process. (This is in contrast with the behavior prior to OpenVMS Version 7.0 of starving all threads equally).
• There are also implications for controlling AST delivery. $SETAST generates an upcall similar to the one for AST delivery. This allows the Threads Library to note the request by a thread to block (or unblock) AST delivery. When a thread has requested that ASTs be blocked, it will not receive delivery of any per-thread ASTs; nor will the process receive delivery of any process ASTs. This is, in effect, the behavior prior to OpenVMS Version 7.0, except that a second thread cannot undo a block requested by a previous thread. Avoid using any mechanism other than $SETAST to block ASTs; it will interfere with the process as a whole and may produce undesirable results.
• Another implication is that a thread can be executing on one virtual processor at the same time that an AST is executing on another virtual processor. In general, this should not pose a significant problem for multithreaded applications. Such applications should have already minimized their AST use, since ASTs and threads can be difficult to use together reliably.
  In addition, AST routines should already be performing only atomic operations, since thread synchronization is not available to code executing at AST level. Any "legacy" code (such as a nonthreaded application using threaded libraries) is executed in the initial thread, where the normal assumptions about AST delivery are maintained. If a piece of code cannot tolerate concurrent execution with an AST routine, it should disable AST delivery during its execution.

In OpenVMS Alpha Version 7.0 and later, with few exceptions a blocking system service call is thread synchronous---that is, only the calling thread is blocked. The exceptions are services that do not block in user mode and services that set common event flags. (See also Section B.12.8.)

When a thread calls a system service that must block, the OpenVMS scheduler makes an upcall to allow the Threads Library to schedule another user thread to execute. Therefore, only the calling thread is blocked, all other threads are unaffected, and the process continues running. When the service completes, the thread is awakened by means of another upcall, and the Threads Library schedules it to run again at the thread's next opportunity.

This applies to all "W" forms of system services, for example, $QIOW, $END_TRANSW, and $GETJPIW. Additionally, this applies to the following event flag services: $WAITFR, $WFLAND, and $WFLOR.

B.12.7 $HIBER and $WAKE

$HIBER and $WAKE result in upcalls to the Threads Library. When a user thread calls $HIBER, only that thread is blocked; all other threads continue running. The blocking thread is immediately unscheduled and another thread is scheduled to run instead. When a thread (or another process) calls $WAKE, all hibernating threads are awakened.

Prior to OpenVMS Version 7.0, a thread that called a $HIBER (or called a library routine that eventually resulted in a call to $HIBER) would cause the whole process to hibernate for a brief period whenever that thread was scheduled to "run." Also, with multiple threads in calls to $HIBER simultaneously, there was no reliable way to wake the threads (or a specific thread); the next hibernating thread to be scheduled would awaken, and any other threads would continue to sleep.

In OpenVMS Alpha Version 7.0 and later, these problems have been resolved. However, this new behavior has some other effects. For instance, hibernation-based services, such as LIB$WAIT and the C RTL sleep() routine, may be prone to premature completion. If the service does not validate its wakeup (that is, ensure that enough time has passed or that there is some other reason for it to return), then it will be prone to this problem, as are the above services, since they do not perform such wake-up validation. (The sleep() routine does this deliberately to mimic the ANSI C required behavior of returning when interrupted by a signal. Though OpenVMS does not have UNIX signals, an asynchronous $WAKE is similar in intent.)

B.12.8 Event Flags

All event flags are shared by all threads in the process. Therefore, it is possible for different threads' use of the same event flag to cause interference.

If two threads use the same event flag in calls to different system services, whichever service completes first will cause both threads to awaken, even though the other service has not completed. This situation can be resolved by specifying an I/O status block (IOSB) for those system services that use them. When an IOSB is present, the blocked thread will not be awakened when the event flag is set, unless the IOSB has also been written.

A Threads Library process is rarely in LEF state. In general, instead of blocking for an event flag wait, the Threads Library schedules another thread to be run. However, if no threads are available, the Threads Library schedules a "null" thread, which places the virtual processor in HIB state until it is needed to execute a thread.

There are several interactions with the OpenVMS operating system that should be noted:

• Like system service calls, pagefault waits are thread synchronous. When a thread incurs a "hard" pagefault (reading the page from disk), an upcall to the Threads Library takes place, and the Threads Library places the thread in a blocked state. The Threads Library schedules another thread to run in its place.
  When the pagefault resolution is complete, another upcall occurs, and the Threads Library schedules it to run at its next opportunity. It is possible for multiple threads to take faults on the same page at approximately the same time. Each thread is blocked, in turn, and becomes unblocked when the page becomes valid.
• Most OpenVMS system services cannot themselves support being called by multiple threads concurrently. Therefore, calls to OpenVMS system services are serialized using a mechanism called the inner-mode semaphore. If one thread attempts to call a system service while another thread is in the middle of calling a system service, the second thread is blocked by an upcall until the first thread completes its service call.
• Threads Library timeslicing changed slightly for OpenVMS Alpha Version 7.0 and later. Prior to Version 7.0, the Threads Library timeslicer was implemented using an OpenVMS timer. This caused a Threads Library scheduler AST to be delivered to the process at regular wall-clock time intervals. While running on wall-clock time was a necessary evil (to support the interruption of system service blocks), this timeslice mechanism had several drawbacks.
  In OpenVMS Version 7.0 and later, timeslicing is implemented as an upcall to the Threads Library that is delivered after the thread has consumed a sufficient amount of CPU time. Thus, when no threads are running, no timeslicing takes place.

In multithreaded processes, image exit occurs as follows: $EXIT does not immediately invoke exit handler routines. Instead, it results in an upcall that causes the Threads Library to schedule a special thread to execute the exit-handler routines. $EXIT then calls pthread_exit() to terminate the calling thread. This allows the calling thread to release any resources that it might be holding.

To avoid possible deadlocks, the exit-handler routines are executed in a separate thread. For example, if a thread calls $EXIT while holding a mutex that is required by an exit-handler routine, then that routine causes the thread to block forever, as it waits for a mutex that it already holds. Because the exit-handler routine executes in a separate thread, it can block while the thread holding the mutex cleans up.

$FORCEX works in an analogous fashion. Instead of invoking $EXIT directly, it causes an upcall that allows the Threads Library to release the exit-handler thread.

DCL Ctrl/Y continues to work as it always has on multithreaded applications. However, typing EXIT or issuing any other command that invokes a new image causes the $FORCEX upcall. While this is an improvement in many cases over the behavior prior to OpenVMS Version 7.0, it does not guarantee that the multithreaded application will exit.

For example, if the application is deadlocked, holding a resource required by one of the exit handler's routines, the application will continue to hang, even after typing Ctrl/Y and EXIT. In these cases, type Ctrl/Y and STOP to terminate the application without running exit handlers. Note that doing so causes the application to be unable to clean up, and it may leave data files and the terminal in an inconsistent state.

B.12.11 SYSGEN Parameter MULTITHREAD

The SYSGEN parameter MULTITHREAD limits the maximum number of kernel threads per process. It is set by AUTOGEN to the number of CPUs on the system. If MULTITHREAD is set to zero (0), two-level scheduling support is disabled, and the Threads Library reverts to its behavior prior to OpenVMS Version 7.0---that is, no upcalls can occur, and it does not use all processors in multiprocessor systems.

B.12.12 Process Control System Services and DCL Commands

OpenVMS system services and DCL commands are either process based or operate on a per-thread basis. This section identifies several system services on this basis.

B.12.12.1 Process-Level System Services

The following system services continue to be process based: $SUSPEND, $RESUME, and $DELPRC. These services will operate on an entire process; they are not thread based. For example, $SUSPEND issued by a thread will suspend all of the virtual processors in process, not just the calling thread.

Under OpenVMS Version 7.0 or later, it is possible, such as when at a breakpoint in the debugger, to see all but one of your kernel threads in SUSP state. This effect is a part of the debugging support and is not the result of calling $SUSPND.

B.12.12.2 Kernel-Level System Services

The following system services now operate on a per-thread basis: $HIBER, $SCHDWK, and $SYNCH. These services will not operate on an entire process; they are thread based. For example, $HIBER will cause the calling thread to become inactive but will not affect other threads in the process.

B.12.12.3 DCL Commands

The following DCL commands operate as indicated:

• STOP/IDENTIFICATION---This command continues to work on a process basis.
• SET PROCESS---This command continues to work on a process basis except for SET PROCESS/PRIORITY.
• SET PROCESS/PRIORITY---This command now sets the priority of a kernel thread. Avoid setting different priorities for kernel threads in the same process. Refer to Section B.12.4 for more information.

Previous releases of the POSIX for OpenVMS layered product had very limited interoperability with the Threads Library. Under OpenVMS Version 7.0 and later, using the Threads Library with the POSIX for OpenVMS layered product is not supported.