HP OpenVMS Systems Documentation

Starts to collect spinlock information a longer period of time than will fit into the trace buffer.

Format

SPL START COLLECT [/SPINLOCK=spinlock|/ADDRESS=n]

Parameters

None.

Qualifiers

/SPINLOCK=spinlock

Specifies the tracing of a specific spinlock, for example, /SPINLOCK=LCKMGR or /SPINLOCK=SCHED.

/ADDRESS=n

Specifies the tracing of a specific spinlock by address.

Description

The SPL START COLLECT command starts a collection of spinlock information for a longer period of time than will fit into the trace buffer. You need to enable spinlock tracing before a spinlock collection can be started. On a system with heavy activity, the trace buffer typically can only hold a relatively small time window of spinlock information. In order to collect spinlock information over a longer time period, a collection can be started. The collection tries to catch up with the running trace index and save the spinlock information into a balanced tree within the virtual address space of the process performing the spinlock collection. Either use the name of a static spinlock, or supply the address of a dynamic spinlock, for which information should be gathered.

The trace entries are kept in the trace buffer, which is allocated from S2 space, hence there is no disruption, if tracing is started from within SDA and then the user exits from SDA. However, for the longer period data collection, the information is kept in process-specific memory, thus a user needs to stay within SDA; otherwise the data collection is automatically terminated by SDA's image rundown. You can collect data for two or more spinlocks simultaneously, by using a separate process for each collection.

Examples

SDA> SPL START COLLECT
Use /SPINLOCK=name or /ADDRESS=n to specify which spinlock info needs to be collected...
      

This example shows that you need to supply either a spinlock name of a static spinlock, or the address of a dynamic spinlock, if you want to collect information over a long period of time.

SDA> SPL START COLLECT/SPINLOCK=LCKMGR
      

This example shows the command line to start to collect information on the usage of the LCKMGR spinlock.

Enables spinlock tracing.

Format

SPL START TRACE [/[NO]SPINLOCK=spinlock|/[NO]FORKLOCK=forklock
|/BUFFER=pages|/[NO]ACQUIRE|
|/[NO]RELEASE|/[NO]WAIT|/[NO]FRKDSPTH
|/[NO]FRKEND|/CPU=n]

Parameters

None.

Qualifiers

/SPINLOCK=spinlock
/NOSPINLOCK

The /SPINLOCK=spinlock qualifier specifies the tracing of a specific spinlock, for example, /SPINLOCK=LCKMGR or /SPINLOCK=SCHED.

The /NOSPINLOCK qualifier disables spinlock tracing and does not collect any spinlock data. If omitted, all spinlocks are traced.

/FORKLOCK=forklock
/NOFORKLOCK

The /FORKLOCK=forklock qualifier specifies the tracing of a specific forklock, for example, /FORKLOCK=IOLOCK8 or /FORKLOCK=IPL8.

The /NOFORKLOCK qualifier disables forklock tracing and does not collect any forklock data. If omitted, all forks are traced.

/BUFFER=pages

Specifies the size of the trace buffer (in Alpha page units). It defaults to 128 pages, which is equivalent to 1MB, if omitted.

/ACQUIRE
/NOACQUIRE

The /ACQUIRE qualifier traces any spinlock acquisitions. This is the default.

The /NOACQUIRE qualifier ignores any spinlock acquisitions.

/RELEASE
/NORELEASE

The /RELEASE qualifier traces any spinlock releases. This is the default.

The /NORELEASE qualifier ignores any spinlock releases.

/WAIT
/NOWAIT

The /WAIT qualifier traces any spinwait operations. This is the default.

The /NOWAIT qualifier ignores any spinwait operations.

/FRKDSPTH
/NOFRKDSPTH

The /FRKDSPTH qualifier traces all invocations of fork routines within the fork dispatcher. This is the default.

The /NOFRKDSPTH qualifier ignores all of the /FRKDSPTH operations.

/FRKEND
/NOFRKEND

The /FRKEND qualifier traces all returns from fork routines within the fork dispatcher. This is the default.

The /NOFRKEND qualifier ignores all of the operations of the /FRKEND qualifier.

/CPU=n

Specifies the tracing of a specific CPU only, for example, /CPU=5 or /CPU=PRIMARY. By default, all CPUs are traced.

Description

The SPL START TRACE command enables spinlock and fork tracing. By default all spinlocks and forks are traced and a 128 page (1MByte) trace buffer is allocated and used as a ring buffer.

Examples

SDA> SPL START TRACE/BUFFER=1000
Tracing started... (Spinlock = 00000000, Forklock = 00000000)
      

This example shows how to enable a tracing for all spinlock and forklock operations into a 8 MByte trace buffer.

SDA> SPL START TRACE/CPU=PRIMARY/SPINLOCK=SCHED /NOFORKLOCK
Tracing started... (Spinlock = 810AF600, Forklock = 00000000)
      

This example shows how to trace only SCHED spinlock operations on the primary CPU.

SDA> SPL START TRACE /NOSPINLOCK /FORKLOCK=IPL8
Tracing started... (Spinlock = 00000000, Forklock = 863A4C00)
      

This example shows how to trace only fork operations to IPL8.

Stops the spinlock collection, but does not stop spinlock tracing.

Format

SPL STOP COLLECT

Parameters

None.

Qualifiers

None.

Description

The SPL STOP COLLECT command stops the data collection, but does not affect tracing. This allows the user to start another collection for a different spinlock during the same trace run.

Example

SDA> SPL STOP COLLECT
      

Disables spinlock tracing, but it does not deallocate the trace buffer.

Format

SPL STOP TRACE

Parameters

None.

Qualifiers

None.

Description

The SPL STOP TRACE command stops tracing, but leaves the trace buffer allocated for further analysis.

Example

SDA> SPL STOP TRACE
Tracing stopped...
      

Unloads the SPL$DEBUG execlet and performs cleanup. Tracing is automatically disabled and the trace buffer deallocated.

Format

SPL UNLOAD

Parameters

None.

Qualifiers

None.

Description

The SPL UNLOAD command disables the tracing or collection functionality with a delay to a state of quiescence. This ensures that all pending trace operations in progress have finished before the trace buffer is deallocated. Finally the SPL UNLOAD command unloads the SPL$DEBUG execlet.

Example

SDA> SPL UNLOAD
SPL$DEBUG unload status = 00000001
      

This chapter describes how to write, debug, and invoke an SDA Extension. This chapter also describes the routines available to an SDA Extension.

7.1 Introduction

When analysis of a dump file or a running system requires intimate knowledge of data structures that are not known to the System Dump Analyzer, the functionality of SDA can be extended by the addition of new commands into which the necessary knowledge has been built. Note that in this description, whenever a reference is made to accessing a dump file (ANALYZE/CRASH_DUMP), this also includes accessing memory in the running system (ANALYZE/SYSTEM).

For example, a user-written device driver allocates nonpaged pool and records additional data about the device there (logging different types of I/O, perhaps), and a pointer to the new structure is saved in the device-specific extension of the UCB. After a system crash, the only way to look at the data from SDA is to do the following:

• Invoke the SDA command DEFINE to define a new symbol (for example, UCB$L_FOOBAR) whose value is the offset in the UCB of the pointer to the new structure.
• Invoke the SDA commands "SHOW DEVICE <device>" and "FORMAT UCB" to obtain the address of the nonpaged pool structure.
• Invoke the SDA command "EXAMINE <address>;<length>" to display the contents of the data in the new nonpaged pool structure as a series of hexadecimal longwords.
• Decode manually the contents of the data structure from this hexadecimal dump.

An SDA extension that knows the layout of the nonpaged pool structure, and where to find the pointer to it in the UCB, could output the data in a formatted display that alerts the user to unexpected data patterns.

7.2 General Description

The following discussion uses an example of an SDA extension that invokes the MBX command to output a formatted display of the status of the mailbox devices in the system. The source file, MBX$SDA.C, is provided in SYS$EXAMPLES.

An SDA extension consists of a shareable image, in this case MBX$SDA.EXE, either located in the directory SYS$LIBRARY or found by translating the logical name MBX$SDA. It contains two universal symbols: SDA$EXTEND, the entry point; and SDA$EXTEND_VERSION, the address of a longword that contains the version of the interface used (in the format of major/minor ident), which allows SDA to confirm it has activated a compatible extension. The image contains at least two modules: MBX$SDA, the user-written module that defines the two symbols and provides the code and data necessary to produce the desired formatted output; and SDA_EXTEND_VECTOR, which provides jackets for all of the callable SDA routines, and is found in SYS$LIBRARY:VMS$VOLATILE_PRIVATE_INTERFACES.OLB. The user-written portion can be split into multiple modules.

Whenever SDA receives an unrecognized command, like "SDA> MBX", it attempts to activate the shareable image MBX$SDA at the SDA$EXTEND entry point. If you choose a command name that matches the abbreviation of an existing command, SDA can be forced to activate the extension using the "DO" command. For example, if you had an SDA extension called VAL$SDA, you could not activate it with a command like "SDA> VAL" as SDA would interpret that as an abbreviation of its VALIDATE command. But VAL$SDA can be activated by issuing "SDA> DO VAL".

With or without the "DO" prefix, the rest of the command line is passed to the extension; it is up to the extension to parse it. The example extension MBX$SDA includes support for commands of the form "SDA> MBX SUMMARY" and "SDA> MBX <address>" to demonstrate this. If the extension is invoked with no arguments, it should do no more than display a simple announcement message, or prompt for input. This assists in the debugging of the extension, as described in Section 7.4.

7.3 Detailed Description

This section describes how to compile, link, and invoke an SDA extension. It also describes the contents of an SDA extension.

7.3.1 Compiling and Linking an SDA Extension

$cc mbx$sda + alpha$library:sys$lib_c /library
$link /share -
                mbx$sda.obj, -
                alpha$library:vms$volatile_private_interfaces /library, -
                sys$input /option
        symbol_vector = (sda$extend=procedure)
        symbol_vector = (sda$extend_version=data)

You can invoke the SDA extension as follows:

$define mbx$sda sys$disk:[]mbx$sda
$analyze /system
SDA>mbx summary
SDA>mbx <address>

At a minimum, the user-written module must contain:

• #include statements for DESCRIP.H and SDA_ROUTINES.H
• The global variable SDA$EXTEND_VERSION, initialized as follows:

  int sda$extend_version = SDA_FLAGS$K_VERSION;

• The routine SDA$EXTEND (prototype follows)

Optionally, the user-written module may also contain the statement:

        #define __NEW_STARLET

You should use this option because it provides type checking of function arguments and gives consistency in casing and naming conventions.

The entry point in the user-written module, SDA$EXTEND, is called as a routine with three arguments and no return value. The declaration is as follows:

        void sda$extend (
                int *transfer_table,
                struct dsc$descriptor_s *cmd_line,
                SDA_FLAGS sda_flags)

The arguments in this code example have the following meanings:

The first executable statement of the routine must be to copy TRANSFER_TABLE to SDA$VECTOR_TABLE (which is declared in SDA_ROUTINES.H):

        sda$vector_table = transfer_table;

If this is not done, you cannot call any of the routines described below. Any attempts to call the routines receive a status return of SDA$_VECNOTINIT. (For routines defined not to return a status, this value can be found only by examining R0.)

The next statement should be one to establish a condition handler, as it is often difficult to track down errors in extensions such as access violations because the extension is activated dynamically with LIB$FIND_IMAGE_SYMBOL. A default condition handler, SDA$COND_HANDLER, is provided that outputs the following information in the event of an error:

• The error condition
• The VMS version
• A list of activated images, with start and end virtual addresses
• The signal array and register dump
• The current call frame chain

You can establish this condition handler as follows:

        lib$establish (sda$cond_handler);

Thus, a minimal extension would be:

        #define __NEW_STARLET 1
        #include <descrip.h>
        #include <sda_routines.h>

        int sda$extend_version = SDA_FLAGS$K_VERSION;

        void sda$extend (int *transfer_table,
                         struct dsc$descriptor_s *cmd_line,
                         SDA_FLAGS sda_flags)
          {
          sda$vector_table = transfer_table;
          lib$establish (sda$cond_handler);

          sda$print ("hello, world");
          return;
          }

In addition to the "after-the-fact" information provided by the condition handler, you can debug SDA extensions using the OpenVMS Debugger. A second copy of the SDA image, SDA_DEBUG.EXE, is provided in SYS$SYSTEM. By defining the logical name SDA to reference this image, you can debug SDA extensions as follows:

• Compile your extension /DEBUG/NOOPT and link it /DEBUG.
• Define logical names for SDA and the extension, and invoke SDA.
• Type GO at the initial DBG> prompt.
• Invoke the extension with no argument at the initial SDA> prompt.
• Return control to Debug at the next prompt (either from SDA or the extension).
• Use Debug commands to set breakpoints, and so on, in the extension and then type GO.
• Invoke the extension, providing the necessary arguments.

An example of the previous procedures is as follows:

        $ cc /debug /noopt mbx$sda + alpha$library:sys$lib_c /library
        $ link /debug /share -
                mbx$sda.obj, -
                alpha$library:vms$volatile_private_interfaces /library, -
                sys$input /option
        symbol_vector = (sda$extend=procedure)
        symbol_vector = (sda$extend_version=data)
        $ !
        $ define mbx$sda sys$disk:[]mbx$sda
        $ define sda sda_debug
        $ analyze /system
        ...
        DBG> go
        ...
        SDA> mbx
        MBX commands: 'MBX SUMMARY' and 'MBX <address>'
        SDA>
        ^C <CR>
        DBG> set image mbx$sda
        DBG> set language c
        DBG> set break /exception
        DBG> go
        SDA> mbx summary
        ...
        SDA> mbx <address>
        ...
        %DEBUG-I-DYNMODSET, setting module MBX$SDA
        %SYSTEM-E-INVARG, invalid argument
        ...
        DBG>

The user-written routine may call SDA routines to accomplish any of the following tasks:

• Read the contents of memory locations in the dump.
• Translate symbol names to values and vice-versa, define new symbols, and read symbol table files.
• Map an address to the activated image or executive image that contains that address.
• Output text to the terminal, with page breaks, page headings, and so on (and which is output to a file if the SDA commands SET OUTPUT or SET LOG have been used).
• Allocate and deallocate dynamic memory.
• Validate queues/lists.
• Format data structures.
• Issue any SDA command.

The full list of available routines is as follows:

The details of all these routines follow. But there are some points to be aware of in using them:

• There are three different routines available to read the contents of memory locations in the dump: SDA$TRYMEM, SDA$GETMEM, and SDA$REQMEM. They are used as follows:
  SDA$TRYMEM is called from both SDA$GETMEM and SDA$REQMEM as the lower-level routine that actually does the work. SDA$TRYMEM returns success/failure status in R0, but does not signal any errors. Use it directly when you expect that the location being read is inaccessible. The caller of SDA$TRYMEM will handle this situation by checking the status returned by SDA$TRYMEM.
  SDA$GETMEM signals a warning when any error status is returned from SDA$TRYMEM. Signaling a warning will print out a warning message, but does not abort the SDA command in progress. You should use this routine when you expect the location to be read to be accessible. This routine does not prevent the command currently being executed from continuing. The caller of SDA$GETMEM must allow for this by checking the status returned by SDA$GETMEM.
  SDA$REQMEM signals an error when any error status is returned from SDA$TRYMEM. Signaling an error will print out an error message, abort the SDA command in progress and return to the "SDA>" prompt. You should use this routine when you expect the location to be read to be accessible. This routine will prevent the command currently being executed from continuing. The caller of SDA$REQMEM will not resume if an error occurs.
• You should use only the routines provided to output text. Do not use printf() or any other standard routine. If you do, the SDA commands SET OUTPUT and SET LOG will not produce the expected results. Do not include control characters in output (except tab); in particular, avoid <CR>, <LF>,<FF>, and the FAO directives that create them. Use the FAO directive !AF when contents of memory returned by SDA$TRYMEM, and so on, are being displayed directly, because embedded control characters will cause undesirable results. For example, displaying process names or resource names that contain particular control characters or escape sequences can lock up the terminal.
• You should use only the routines provided to allocate and deallocate dynamic memory. Do not use malloc() and free(). Where possible, allocate dynamic memory once, the first time the extension is activated, and deallocate it only if it needs to be replaced by a larger allocation. Because SDA commands can be interrupted by invoking another command at the "Press return for more" prompt, it is very easy to cause memory leaks.
• Some routines expect 32-bit pointers, and others expect 64-bit pointers. At first this not may appear to be logical, but in fact it is. All code and data used by SDA and any extensions must be in P0 or P1 space, as SDA does not need to (and does not) use P2 space for local data storage. However, addresses in the system dump (or running system, in the case of ANALYZE/SYSTEM) are 64-bit addresses, and SDA must provide access to all locations in the dump.