HP OpenVMS System Analysis Tools Manual
Previous Contents Index

Chapter 5
SDA CLUE Extension

The SDA CLUE command invokes the Crash Log Utility Extractor, which captures specific crash dump information and, upon system reboot, preserves it in a file with the following naming scheme: 


     CLUE$nodename_ddmmyy_hhmm.LIS

You enter CLUE extension commands at the SDA prompt. For example: 


     SDA> CLUE CONFIG

You can get full help on CLUE by entering HELP CLUE at the SDA> prompt.

5.1 Overview of SDA CLUE Extension

SDA CLUE (Crash Log Utility Extractor) commands automate the analysis of crash dumps and maintain a history of all fatal bugchecks on either a standalone or cluster system. You can use SDA CLUE commands in conjunction with SDA to collect and decode additional dump file information not readily accessible through standard SDA commands. SDA CLUE extension commands can summarize information provided by certain standard SDA commands and provide additional detail for some SDA commands. For example, SDA CLUE extension commands can quickly provide detailed extended QIO processor (XQP) summaries. You can also use SDA CLUE commands interactively on a running system to help identify performance problems.

You can use all CLUE commands when analyzing crash dumps; the only CLUE commands that are not allowed when analyzing a running system are CLUE CRASH, CLUE ERRLOG, CLUE HISTORY, and CLUE STACK.

When you reboot the system after a system failure, you automatically invoke SDA by default. To facilitate better crash dump analysis, SDA CLUE commands automatically capture and archive summary dump file information in a CLUE listing file.

A startup command procedure initiates commands that do the following:

The CLUE HISTORY command adds a one-line summary entry to a history file and saves the following output from SDA CLUE commands in the listing file:

The contents of this CLUE list file can help you analyze a system failure. If these files accumulate more space than the threshold allows (default is 5000 blocks), the oldest files are deleted until the threshold limit is reached. You can also customize this threshold using the CLUE$MAX_BLOCKS logical name.

For additional information on the contents of the CLUE listing file, see the reference section on CLUE HISTORY.

It is important to remember that CLUE$nodename_ddmmyy_hhmm.LIS contains only an overview of the crash dump and does not always contain enough information to determine the cause of the crash. The dump itself should always be saved using the procedures described in Section 2.2.2 and Section 2.2.4.

To inhibit the running of CLUE at system startup, define the logical CLUE$INHIBIT in the SYLOGICALS.COM file as /SYS TRUE.

5.2 Displaying Data with CLUE

To invoke a CLUE command, enter the command at the SDA prompt. For example: 


SDA> CLUE CONFIG

5.3 Using CLUE with DOSD

DOSD (Dump Off System Disk) allows you to write the system dump file to a device other than the system disk. For SDA CLUE to be able to correctly find the dump file to be analyzed after a system crash, you need to perform the following steps:

  1. Modify the command procedure SYS$MANAGER:SYCONFIG.COM to add the system logical name CLUE$DOSD_DEVICE to point to the device where the dump file resides. You need to supply only the physical or logical device name without a file specification.
  2. Modify the command procedure SYS$MANAGER:SYCONFIG.COM to mount systemwide the device where the dump file resides. Otherwise, SDA CLUE cannot access and analyze the dump file.

In the following example, the dump file has been placed on device $3$DUA25, which has the label DMP$DEV. You need to add the following commands to SYS$MANAGER:SYCONFIG.COM: 


$ MOUNT/SYSTEM/NOASSIST $3$DUA25: DMP$DEV DMP$DEV
$ DEFINE/SYSTEM CLUE$DOSD_DEVICE DMP$DEV

5.4 SDA CLUE Extension Commands

The following pages describe the SDA CLUE extension commands.

CLUE CALL_FRAME (Alpha Only)

Displays key information, such as the PC of the caller, from the active call frames at the time of the crash.

Format

CLUE CALL_FRAME [/CPU [cpu-id|ALL]
|/PROCESS [/ADDRESS=n|INDEX=n
|/IDENTIFICATION=n|process-name|ALL]]

Parameters

ALL

When used with /CPU, it requests information about all CPUs in the system. When used with /PROCESS, it requests information about all processes that exist in the system.

cpu-id

When used with /CPU, it gives the number of the CPU for which information is to be displayed. Use of the cpu-id parameter causes the CLUE CALL_FRAME command to perform an implicit SET CPU command, making the indicated CPU the current CPU for subsequent SDA commands.

process-name

When used with /PROCESS, it gives the name of the process for which information is to be displayed. Use of the process-name parameter, the /ADDRESS qualifier, the /INDEX qualifier, or the /IDENTIFICATION qualifier causes the CLUE CALL_FRAME command to perform an implicit SET PROCESS command, making the indicated process the current process for subsequent SDA commands. You can determine the names of the processes in the system by issuing a SHOW SUMMARY command.

The process-name can contain up to 15 letters and numerals, including the underscore (_) and dollar sign ($). If it contains any other characters, you must enclose the process-name in quotation marks (" ").

Qualifiers

/ADDRESS=n

Specifies the PCB address of the desired process when used with CLUE CALL_FRAME/PROCESS.

/CPU [cpu-id|ALL]

Indicates that the call frame for a CPU is required. Specify the CPU by its number or use ALL to indicate all CPUs.

/IDENTIFICATION=n

Specifies the identification of the desired process when used with CLUE CALL_FRAME/PROCESS.

/INDEX=n

Specifies the index of the desired process when used with CLUE CALL_FRAME/PROCESS.

/PROCESS [process-name|ALL]

Indicates that the call frame for a process is required. The process should be specified with either one of the qualifiers /ADDRESS, /IDENTIFICATION, or /INDEX, or by its name, or by using ALL to indicate all processes.

Description

The CLUE CALL_FRAME command displays call chain information for a process or a CPU. The process context calls work on both the running system and dump file; the CPU context calls only on dump files.

If neither /CPU nor /PROCESS is specified, the parameter (CPU-id or process-name) is ignored and the call frame for the SDA current process is displayed.

Examples

#1 

SDA>CLUE CALL/PROCESS IPCACP
Call Chain:   Process index: 000B   Process name: IPCACP  PCB: 8136EF00
-----------------------------------------------------------------------
Procedure Frame  Procedure Entry                          Return Address
---------------  ----------------------------------       ---------------------------
7FFA1CA0  Null   800C8C90  SCH$WAIT_PROC_C                 
7FFA1D00  Stack  800D9250  SYS$HIBER_C                    0003045C  IPCACP+0003045C
7FFA1D50  Stack  00030050  IPCACP+00030050                800D11C8  EXE$CMKRNL_C+000D8
7FFA1E60  Null   800B6120  EXE$BLDPKTSWPR_C                
7FFA1E78  Null   800B6120  EXE$BLDPKTSWPR_C                
7FFA1EC0  Null   80248120  NSA$CHECK_PRIVILEGE_C                      
7FFA1F00  Null   80084640  EXE$CMODEXECX_C                 
7FFA1F70  Stack  800D10F0  EXE$CMKRNL_C                   80084CC8  EXE$CMODKRNL_C+00198
7B01FAB0  Stack  00030010  IPCACP+00030010                83EA3454  SYS$IMGSTA_C+00154
7B01FB10  Stack  83EA3300  SYS$IMGSTA_C                   83D99CC4  EXE$PROC_IMGACT_C+00384
7B01FBA0  Stack  83D99BA0  EXE$PROC_IMGACT_C+00260        83D99B9C  EXE$PROC_IMGACT_C+0025C

In this example, the CLUE CALL_FRAME command displays the call frame from the process IPCACP.

#2 

SDA>CLUE CALL/CPU ALL
Call Chain:   Process index: 0000   Process name: NULL    PCB: 827377C0   (CPU 0)
---------------------------------------------------------------------------------
Procedure Frame  Procedure Entry                          Return Address
---------------  ----------------------------------       ---------------------------
8F629D28  Null   80205E00  SYS$SCS+05E00                   
8F629D68  Null   8020A850  SCS$REC_MSGREC_C                
8F629D98  Null   914A5340  SYS$PBDRIVER+07340              
8F629DB8  Null   914A4FD0  SYS$PBDRIVER+06FD0              
8F629DE0  Stack  914AACF0  SYS$PBDRIVER+0CCF0             914AE5CC  SYS$PBDRIVER+105CC
8F629E50  Stack  914AE418  SYS$PBDRIVER+10418             800503B0  EXE_STD$QUEUE_FORK_C+00350
8F629F88  Null   800E95F4  SCH$WAIT_ANY_MODE_C             
8F629FD0  Stack  800D0F80  SCH$IDLE_C                     800E92D0  SCH$INTERRUPT+00BB0
Call Chain:   Process index: 0000   Process name: NULL    PCB: 827377C0   (CPU 2)
---------------------------------------------------------------------------------
Procedure Frame  Procedure Entry                          Return Address
---------------  --------------------------------         ---------------------------
90FCBF88  Null   800E95F4  SCH$WAIT_ANY_MODE_C             
90FCBFC8  Null   800E95F4  SCH$WAIT_ANY_MODE_C            
90FCBFD0  Stack  800D0F80  SCH$IDLE_C                     800E92D0  SCH$INTERRUPT+00BB0
Call Chain:   Process index: 0000   Process name: NULL    PCB: 827377C0   (CPU 6)
---------------------------------------------------------------------------------
Procedure Frame  Procedure Entry                          Return Address
---------------  ------------------------------           ---------------------------
90FCBF88  Null   800E95FA  SCH$WAIT_ANY_MORE_c             
90FD9F88  Null   800E95F4  SCH$WAIT_ANY_MODE_C             
90FD9FD0  Stack  800D0F80  SCH$IDLE_C                     800E92D0  SCH$INTERRUPT+00BB0

In this example, CLUE/CPU ALL shows the call frame for all CPUs.

CLUE CLEANUP

Performs housekeeping operations to conserve disk space.

Format

CLUE CLEANUP

Parameters

None.

Qualifiers

None.

Description

CLUE CLEANUP performs housekeeping operations to conserve disk space. To avoid filling up the system disk with listing files generated by CLUE, CLUE CLEANUP is run during system startup to check the overall disk space used by all CLUE$*.LIS files.

If the CLUE$COLLECT:CLUE$*.LIS files occupy more space than the logical CLUE$MAX_BLOCKS allows, then the oldest files are deleted until the threshold is reached. If this logical name is not defined, a default value of 5,000 disk blocks is assumed. A value of zero disables housekeeping and no check on the disk space is performed.

Example


SDA>  CLUE CLEANUP
%CLUE-I-CLEANUP, housekeeping started...
%CLUE-I-MAXBLOCK, maximum blocks allowed 5000 blocks
%CLUE-I-STAT, total of 4 CLUE files, 192 blocks.

In this example, the CLUE CLEANUP command displays that the total number of blocks of disk space used by CLUE files does not exceed the maximum number of blocks allowed. No files are deleted.

CLUE CONFIG

Displays the system, memory, and device configurations.

Format

CLUE CONFIG

Parameters

None.

Qualifiers

/ADAPTER

Displays only the part of the system configuration that contains information about the adapters and devices on the system.

/CPU

Displays only the part of the system configuration that contains information about the CPUs.

/MEMORY

Displays only the part of the system configuration that contains information about the layout of physical memory.

Description

CLUE CONFIG displays the system, memory, and device configurations. If no qualifier is specified, the entire system configuration is displayed (memory, CPUs, adapters, and devices), plus additional system information.

Example

CLUE CRASH

Displays a crash dump summary.

Format

CLUE CRASH

Parameters

None.

Qualifiers

None.

Description

CLUE CRASH displays a crash dump summary, which includes the following items:

Example


SDA>  CLUE CRASH
Crash Time:        30-AUG-1996 13:13:46.83 
Bugcheck Type:     SSRVEXCEPT, Unexpected system service exception 
Node:              SWPCTX  (Standalone) 
CPU Type:          DEC 3000 Model 400 
VMS Version:       X6AF-FT2 
Current Process:   SYSTEM 
Current Image:     $31$DKB0:[SYS0.][SYSMGR]X.EXE;1 
Failing PC:        00000000.00030078    SYS$K_VERSION_01+00078 
Failing PS:        00000000.00000003 
Module:            X 
Offset:            00030078 
 
Boot Time:         30-AUG-1996 09:06:22.00 
System Uptime:               0 04:07:24.83 
Crash/Primary CPU: 00/00 
System/CPU Type:   0402 
Saved Processes:   18 
Pagesize:          8 KByte (8192 bytes) 
Physical Memory:   64 MByte (8192 PFNs, contiguous memory) 
Dumpfile Pagelets: 98861 blocks 
Dump Flags:        olddump,writecomp,errlogcomp,dump_style 
Dump Type:         raw,selective 
EXE$GL_FLAGS:      poolpging,init,bugdump 
Paging Files:      1 Pagefile and 1 Swapfile installed 
 
Stack Pointers: 
KSP = 00000000.7FFA1C98   ESP = 00000000.7FFA6000   SSP = 00000000.7FFAC100 
USP = 00000000.7AFFBAD0 
 
General Registers: 
R0  = 00000000.00000000   R1  = 00000000.7FFA1EB8   R2  = FFFFFFFF.80D0E6C0 
R3  = FFFFFFFF.80C63460   R4  = FFFFFFFF.80D12740   R5  = 00000000.000000C8 
R6  = 00000000.00030038   R7  = 00000000.7FFA1FC0   R8  = 00000000.7FFAC208 
R9  = 00000000.7FFAC410   R10 = 00000000.7FFAD238   R11 = 00000000.7FFCE3E0 
R12 = 00000000.00000000   R13 = FFFFFFFF.80C6EB60   R14 = 00000000.00000000 
R15 = 00000000.009A79FD   R16 = 00000000.000003C4   R17 = 00000000.7FFA1D40 
R18 = FFFFFFFF.80C05C38   R19 = 00000000.00000000   R20 = 00000000.7FFA1F50 
R21 = 00000000.00000000   R22 = 00000000.00000001   R23 = 00000000.7FFF03C8 
R24 = 00000000.7FFF0040   AI  = 00000000.00000003   RA  = FFFFFFFF.82A21080 
PV  = FFFFFFFF.829CF010   R28 = FFFFFFFF.8004B6DC   FP  = 00000000.7FFA1CA0 
PC  = FFFFFFFF.82A210B4   PS  = 18000000.00000000 
 
Exception Frame: 
R2  = 00000000.00000003   R3  = FFFFFFFF.80C63460   R4  = FFFFFFFF.80D12740 
R5  = 00000000.000000C8   R6  = 00000000.00030038   R7  = 00000000.7FFA1FC0 
PC  = 00000000.00030078   PS  = 00000000.00000003 
 
Signal Array:                            64-bit Signal Array: 
Arg Count    = 00000005                  Arg Count      =          00000005 
Condition    = 0000000C                  Condition      = 00000000.0000000C 
Argument #2  = 00010000                  Argument #2    = 00000000.00010000 
Argument #3  = 00000000                  Argument #3    = 00000000.00000000 
Argument #4  = 00030078                  Argument #4    = 00000000.00030078 
Argument #5  = 00000003                  Argument #5    = 00000000.00000003 
 
Mechanism Array: 
Arguments    = 0000002C                  Establisher FP = 00000000.7AFFBAD0 
Flags        = 00000000                  Exception FP   = 00000000.7FFA1F00 
Depth        = FFFFFFFD                  Signal Array   = 00000000.7FFA1EB8 
Handler Data = 00000000.00000000         Signal64 Array = 00000000.7FFA1ED0 
R0  = 00000000.00020000   R1  = 00000000.00000000   R16 = 00000000.00020004 
R17 = 00000000.00010050   R18 = FFFFFFFF.FFFFFFFF   R19 = 00000000.00000000 
R20 = 00000000.7FFA1F50   R21 = 00000000.00000000   R22 = 00000000.00010050 
R23 = 00000000.00000000   R24 = 00000000.00010051   R25 = 00000000.00000000 
R26 = FFFFFFFF.8010ACA4   R27 = 00000000.00010050   R28 = 00000000.00000000 
 
System Registers: 
Page Table Base Register (PTBR)                           00000000.00001136 
Processor Base Register (PRBR)                            FFFFFFFF.80D0E000 
Privileged Context Block Base (PCBB)                      00000000.003FE080 
System Control Block Base (SCBB)                          00000000.000001DC 
Software Interrupt Summary Register (SISR)                00000000.00000000 
Address Space Number (ASN)                                00000000.0000002F 
AST Summary / AST Enable (ASTSR_ASTEN)                    00000000.0000000F 
Floating-Point Enable (FEN)                               00000000.00000000 
Interrupt Priority Level (IPL)                            00000000.00000000 
Machine Check Error Summary (MCES)                        00000000.00000000 
Virtual Page Table Base Register (VPTB)                   FFFFFFFC.00000000 
 
Failing Instruction: 
SYS$K_VERSION_01+00078:         LDL             R28,(R28) 
 
Instruction Stream (last 20 instructions): 
SYS$K_VERSION_01+00028:         LDQ             R16,#X0030(R13) 
SYS$K_VERSION_01+0002C:         LDQ             R27,#X0048(R13) 
SYS$K_VERSION_01+00030:         LDA             R17,(R28) 
SYS$K_VERSION_01+00034:         JSR             R26,(R26) 
SYS$K_VERSION_01+00038:         LDQ             R26,#X0038(R13) 
SYS$K_VERSION_01+0003C:         BIS             R31,SP,SP 
SYS$K_VERSION_01+00040:         BIS             R31,R26,R0 
SYS$K_VERSION_01+00044:         BIS             R31,FP,SP 
SYS$K_VERSION_01+00048:         LDQ             R28,#X0008(SP) 
SYS$K_VERSION_01+0004C:         LDQ             R13,#X0010(SP) 
SYS$K_VERSION_01+00050:         LDQ             FP,#X0018(SP) 
SYS$K_VERSION_01+00054:         LDA             SP,#X0020(SP) 
SYS$K_VERSION_01+00058:         RET             R31,(R28) 
SYS$K_VERSION_01+0005C:         BIS             R31,R31,R31 
SYS$K_VERSION_01+00060:         LDA             SP,#XFFE0(SP) 
SYS$K_VERSION_01+00064:         STQ             FP,#X0018(SP) 
SYS$K_VERSION_01+00068:         STQ             R27,(SP) 
SYS$K_VERSION_01+0006C:         BIS             R31,SP,FP 
SYS$K_VERSION_01+00070:         STQ             R26,#X0010(SP) 
SYS$K_VERSION_01+00074:         LDA             R28,(R31) 
SYS$K_VERSION_01+00078:         LDL             R28,(R28) 
SYS$K_VERSION_01+0007C:         BEQ             R28,#X000007 
SYS$K_VERSION_01+00080:         LDQ             R26,#XFFE8(R27) 
SYS$K_VERSION_01+00084:         BIS             R31,R26,R0 
SYS$K_VERSION_01+00088:         BIS             R31,FP,SP

CLUE ERRLOG

Extracts the error log buffers from the dump file and places them into the binary file called CLUE$ERRLOG.SYS.

Format

CLUE ERRLOG [/OLD]

Parameters

None.

Qualifier

/OLD

Dumps the errorlog buffers into a file using the old errorlog format. The default action, if /OLD is not specified, is to dump the errorlog buffers in the common event header format.

Description

CLUE ERRLOG extracts the error log buffers from the dump file and places them into the binary file called CLUE$ERRLOG.SYS.

These buffers contain messages not yet written to the error log file at the time of the failure. When you analyze a failure on the same system on which it occurred, you can run the Error Log utility on the actual error log file to see these error log messages. When analyzing a failure from another system, use the CLUE ERRLOG command to create a file containing the failing system's error log messages just prior to the failure. System failures are often triggered by hardware problems, so determining what, if any, hardware errors occurred prior to the failure can help you troubleshoot a failure.

You can define the logical CLUE$ERRLOG to any file specification if you want error log information written to a file other than CLUE$ERRLOG.SYS.

Note
You need at least DECevent V2.9 to analyze the new common event header (CEH) format file. The old format file can be analyzed by ANALYZE/ERROR or any version of DECevent.

Example


SDA>  CLUE ERRLOG
 
Sequence  Date        Time
--------  ----------- -----------
     128  11-MAY-1994 00:39:31.30
     129  11-MAY-1994 00:39:32.12
     130  11-MAY-1994 00:39:44.83
     131  11-MAY-1994 00:44:38.97 * Crash Entry

In addition to writing the error log buffers into CLUE$ERRLOG.SYS, the CLUE ERRLOG command displays the sequence, date, and time of each error log buffer extracted from the dump file.

CLUE FRU

Outputs the Field Replacement Unit (FRU) table to a file for display by DECevent.

Format

CLUE FRU

Parameters

None.

Qualifiers

None.

Description

The FRU command extracts the FRU table into an output file (CLUE$FRU.SYS), which can then be displayed by DECevent. This command works on the running system, as well as on dump files.

CLUE HISTORY

Updates history file and generates crash dump summary output.

Format

CLUE HISTORY [/qualifier]

Parameters

None.

Qualifier

/OVERRIDE

Allows execution of this command even if the dump file has already been analyzed (DMP$V_OLDDUMP bit set).

Description

This command updates the history file pointed to by the logical name CLUE$HISTORY with a one-line entry and the major crash dump summary information. If CLUE$HISTORY is not defined, a file CLUE$HISTORY.DAT in your default directory will be created.

In addition, a listing file with summary information about the system failure is created in the directory pointed to by CLUE$COLLECT. The file name is of the form CLUE$node_ddmmyy_hhmm.LIS where the timestamp (hhmm) corresponds to the system failure time and not the time when the file was created.

The listing file contains summary information collected from the following SDA commands:

Refer to the reference section for each of these commands to see examples of the displayed information.

The logical name CLUE$FLAG controls how much information is written to the listing file.

If this logical name is undefined, all bits are set by default internally and all information is written to the listing file. If the value is zero, no listing file is generated. The value has to be supplied in hexadecimal form (for example, DEFINE CLUE$FLAG 81 will include the crash dump summary and the XQP cache header information).

If the logical name CLUE$SITE_PROC points to a valid and existing file, it will be executed as the final step of the CLUE HISTORY command (for example, automatic saving of the dump file during system startup). If used, this file should contain only valid SDA commands.

Refer to Chapter 2, Section 2.2.4 for more information on site-specific command files.

CLUE MCHK

This command is obsolete.

Format

CLUE MCHK

Parameters

None.

Qualifiers

None.

Description

The CLUE MCMK command has been withdrawn. Issuing the command produces the following output, explaining the correct way to obtain MACHINECHECK information from a crash dump. 


Please use the following commands in order to extract the errorlog buffers 
from the dumpfile header and analyze the machine check entry: 
 
$ analyze/crash sys$system:sysdump.dmp 
SDA> clue errlog 
SDA> exit 
$ diagnose clue$errlog

Previous Next Contents Index