Reconfigured cluster; 1 member doesn't see disks

From: Adametz, Bluejay <bluejay_at_fujigreenwood.com>
Date: Wed, 29 Aug 2001 07:39:43 -0400

This may take some explaining, so please bear with me...

I did some reconfiguration the other day to allow two clusters to have
access to a single tape library on a SAN, as well as improve reliability.

Each cluster consisted of two ES40s, an 8-port SAN switch, and a
dual-controller HSG80. There was no connection between clusters other than
the usual LAN. Let's call the two clusters A and B, and the individual
members A1, A2, B1, and B2. All the systems are running Tru64 V4.0F &
TruCluster 1.6 production server pk 4. The shared disks on the HSG80 are
used by Oracle as DRD devices.

What I did was to interconnect the two fiber channel switches and re-cable
the nodes so that each cluster had one ES40 and one HSG80 controller going
to each switch. The idea is that should a switch fail, there would be enough
left for us to create a working configuration. These two switches are then
connected to a 3rd switch in another building, to which the tape library is
connected.

To keep the two clusters from getting into each other's disks, I defined
access for the appropriate disks on each HSG80 for the appropriate ES40s.

For cluster A, all this seems to have worked out ok.

For cluster B, everything is working, except that when we attempt to start
cluster services on B2, it complains that it can't access the shared disks.
I've included a sampling of the log messages below. All the services run
fine on B1.

I've checked, and both B1 and B2 report seeing the disks (as well as the
tape library devices) on boot. The SCSI EDT lists the disks. The emx.db file
is the same on both nodes.

Unfortunately, right now the cluster is in production, and I can't do a
whole lot with it. I'm hoping I can get an hour of troubleshooting time
sometime soon. My first idea is to boot each system single-user and just see
if I can do I/O (reads) from each of the disks from each of the members.
That should tell me if I messed up something on the HSG80 or if it's
something higher up (TruCluster?).

My question for y'all is: what else can I look at when I get the system?
(did I mention that I know almost nothing about this cluster stuff, except
what I figured out from reading man pages and looking at files?)

Supporting information follows. The information is verbatim, except that the
node names have been changed for security considerations and clarity.

---
Bluejay Adametz, CFII, A&P	bluejay_at_fujigreenwood.com
Fuji Photo Film, Inc.		+1 864 223 2888 x1369
Greenwood, SC, USA
---
You only think it's hot because the temperature is so high.
Supporting information:
=======================
First, the errors:
------------------
Aug 27 15:11:55 b1 ASE: mcb2 Agent Notice: starting service K5HIST
Aug 27 15:11:55 b1 ASE: mcb1 Agent Notice: starting service K5RQI
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132b'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132c'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132d'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132e'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132f
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132f'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132f
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132g
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132g'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132g
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132h
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132h'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzb132h
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132b'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzc132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132c'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzc132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132d'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzc132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132e'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reserve /dev/rrzc132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132b'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132c'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132d'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132e'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132f
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132f'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132g
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132g'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132h
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132h'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132b'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132c'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132d'
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132e'
Aug 27 15:11:55 b1 ASE: mcb1 Agent Notice: starting service K5RQIDS1
Aug 27 15:11:55 b1 ASE: mcb2 Director Notice: started service K5RQIDS1 on
mcb1
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132c
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132d
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132e
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132f
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132g
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132h
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132b
Aug 27 15:11:55 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132c
Aug 27 15:11:56 b1 ASE: mcb1 Agent Notice: starting service K5HIST
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132d
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132e
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132b
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132b'
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132b
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132b
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132c
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132c'
Aug 27 15:11:56 b1 ASE: mcb2 Director Notice: started service K5RQI on mcb1
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132c
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132c
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132d
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132d'
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132d
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132d
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132e
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132e'
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132e
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132e
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132f
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132f'
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132f
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132f
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132g
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132g'
Aug 27 15:11:56 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132g
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132g
Aug 27 15:11:56 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzb132h
Aug 27 15:11:56 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzb132h'
Aug 27 15:11:57 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzb132h
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzb132h
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132b
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132b'
Aug 27 15:11:57 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132b
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzc132b
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132c
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132c'
Aug 27 15:11:57 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132c
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzc132c
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132d
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132d'
Aug 27 15:11:57 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132d
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzc132d
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device (2)
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: AM can't ping /dev/rrzc132e
Aug 27 15:11:57 b1 ASE: mcb2 Agent Warning: can't reach device
'/dev/rrzc132e'
Aug 27 15:11:57 b1 ASE: mcb2 Agent ***ALERT: possible device failure:
/dev/rrzc132e
Aug 27 15:11:57 b1 ASE: mcb2 Agent Error: can't unreserve device
/dev/rrzc132e
Aug 27 15:11:57 b1 ASE: mcb2 Agent Notice: starting service K5RQIDS2
Aug 27 15:11:57 b1 ASE: mcb2 Director Notice: started service K5RQIDS2 on
mcb2
Aug 27 15:11:57 b1 ASE: mcb2 Director Notice: started service K5HIST on mcb1
The HSG80 configuration:
------------------------
HSG80> show conn
Connection
Unit
   Name      Operating system    Controller  Port    Address    Status
Offset
A11A1            WINNT             OTHER      2      031200   OL other
100
           HOST_ID=2000-0000-C921-7FEE
ADAPTER_ID=1000-0000-C921-7FEE
A11B1            WINNT             THIS       1      031200   OL this
0
           HOST_ID=2000-0000-C921-7FEE
ADAPTER_ID=1000-0000-C921-7FEE
A21A1            WINNT             THIS       1      021200   OL this
0
           HOST_ID=2000-0000-C921-7415
ADAPTER_ID=1000-0000-C921-7415
A21B1            WINNT             OTHER      2      021200   OL other
100
           HOST_ID=2000-0000-C921-7415
ADAPTER_ID=1000-0000-C921-7415
B11A1            WINNT             THIS       1      021500   OL this
0
           HOST_ID=2000-0000-C922-2205
ADAPTER_ID=1000-0000-C922-2205
B11B1            WINNT             OTHER      2      021500   OL other
100
           HOST_ID=2000-0000-C922-2205
ADAPTER_ID=1000-0000-C922-2205
B21A1            WINNT             THIS       1      031500   OL this
0
           HOST_ID=2000-0000-C922-2CB1
ADAPTER_ID=1000-0000-C922-2CB1
B21B1            WINNT             OTHER      2      031500   OL other
100
           HOST_ID=2000-0000-C922-2CB1
ADAPTER_ID=1000-0000-C922-2CB1
HSG80> show unit full
    LUN                                      Uses             Used by
----------------------------------------------------------------------------
--
  D1                                         M1
        LUN ID:      6000-1FE1-0004-76C0-0009-0021-2334-001D
        NOIDENTIFIER
        Switches:
          RUN                    NOWRITE_PROTECT        READ_CACHE
          READAHEAD_CACHE        WRITEBACK_CACHE
          MAXIMUM_CACHED_TRANSFER_SIZE = 32
        Access:
          B11A1, B11B1, B21A1, B21B1
        State:
          ONLINE to this controller
          Reserved
        Size: 35556389 blocks
        Geometry (C/H/S): ( 7000 / 20 / 254 )
  D2                                         M2
        LUN ID:      6000-1FE1-0004-76C0-0009-0021-2334-0020
        NOIDENTIFIER
        Switches:
          RUN                    NOWRITE_PROTECT        READ_CACHE
          READAHEAD_CACHE        WRITEBACK_CACHE
          MAXIMUM_CACHED_TRANSFER_SIZE = 32
        Access:
          B11A1, B11B1, B21A1, B21B1
        State:
          ONLINE to this controller
          Reserved
        Size: 35556389 blocks
        Geometry (C/H/S): ( 7000 / 20 / 254 )
  D101                                       M3
        LUN ID:      6000-1FE1-0004-76C0-0009-0021-2334-0023
        NOIDENTIFIER
        Switches:
          RUN                    NOWRITE_PROTECT        READ_CACHE
          READAHEAD_CACHE        WRITEBACK_CACHE
          MAXIMUM_CACHED_TRANSFER_SIZE = 32
        Access:
          B11A1, B11B1, B21A1, B21B1
        State:
          ONLINE to the other controller
        Size: 35556389 blocks
        Geometry (C/H/S): ( 7000 / 20 / 254 )
  D102                                       M4
        LUN ID:      6000-1FE1-0004-76C0-0009-0021-2334-0026
        NOIDENTIFIER
        Switches:
          RUN                    NOWRITE_PROTECT        READ_CACHE
          READAHEAD_CACHE        WRITEBACK_CACHE
          MAXIMUM_CACHED_TRANSFER_SIZE = 32
        Access:
          B11A1, B11B1, B21A1, B21B1
        State:
          ONLINE to the other controller
        Size: 35556389 blocks
        Geometry (C/H/S): ( 7000 / 20 / 254 )
HSG80>
Boot record, EDT, and emx.info from B1:
---------------------------------------
Aug 27 15:11:39 b1 vmunix: emx0 at pci1 slot 5
Aug 27 15:11:39 b1 vmunix: KGPSA-BC : Driver Rev 1.21 : F/W Rev 3.03A1(1.31)
: wwn 1000-0000-c922-2205
Aug 27 15:11:39 b1 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x21200
assigned tgt id 11 - out of range for CAM
Aug 27 15:11:39 b1 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x31500
assigned tgt id 9 - out of range for CAM
Aug 27 15:11:39 b1 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x31200
assigned tgt id 12 - out of range for CAM
Aug 27 15:11:39 b1 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x21100
assigned tgt id 13 - out of range for CAM
Aug 27 15:11:39 b1 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x31000
assigned tgt id 14 - out of range for CAM
Aug 27 15:11:39 b1 vmunix: scsi16 at emx0 slot 0
Aug 27 15:11:39 b1 vmunix: rzb128 at scsi16 target 0 lun 1 (LID=1) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:39 b1 vmunix: rzc128 at scsi16 target 0 lun 2 (LID=2) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:39 b1 vmunix: rzb132 at scsi16 target 4 lun 1 (LID=3) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:39 b1 vmunix: rzc132 at scsi16 target 4 lun 2 (LID=4) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:39 b1 vmunix: Type 0xc at scsi16 target 5 lun 0 (LID=5) (COMPAQ
DATA ROUTER      1170)
Aug 27 15:11:39 b1 vmunix: changer at scsi16 target 5 lun 1 (LID=6) (DEC
TL810    (C) DEC 2.31)
Aug 27 15:11:40 b1 vmunix: tzc133 at scsi16 target 5 lun 2 (LID=7) (DEC
TZ89     (C) DEC 2150) (Wide16)
Aug 27 15:11:40 b1 vmunix: tzd133 at scsi16 target 5 lun 3 (LID=8) (DEC
TZ89     (C) DEC 2150) (Wide16)
CAM Equipment Device Table (EDT) Information:
    Device: TZ89       Bus: 0, Target: 3, Lun: 0, Type: Sequential Access
    Device: TLZ10      Bus: 2, Target: 5, Lun: 0, Type: Sequential Access
    Device: CDR-8435   Bus: 3, Target: 0, Lun: 0, Type: Read-Only Direct
Access
    Device: HSG80      Bus: 16, Target: 0, Lun: 1, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 0, Lun: 2, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 4, Lun: 1, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 4, Lun: 2, Type: Direct Access
    Device: DATA ROUTER Bus: 16, Target: 5, Lun: 0, Type: Array Controller
    Device: TL810      Bus: 16, Target: 5, Lun: 1, Type: Medium Changer
    Device: TZ89       Bus: 16, Target: 5, Lun: 2, Type: Sequential Access
    Device: TZ89       Bus: 16, Target: 5, Lun: 3, Type: Sequential Access
  emx? tgtid  FC Port Name                     FC Node Name
{   0,    0,  0x0050, 0xe11f, 0x0400, 0xc176,  0x0050, 0xe11f, 0x0400,
0xc076 },
{   0,    1,  0xfd20, 0x6000, 0x2069, 0xe826,  0x0010, 0x6000, 0x2069,
0xe826 },
{   0,    2,  0x0010, 0x0000, 0x22c9, 0xb12c,  0x0010, 0x0000, 0x22c9,
0xb12c },
{   0,    3,  0x0420, 0x6000, 0x2069, 0xe826,  0x0010, 0x6000, 0x2069,
0xe826 },
{   0,    4,  0x0050, 0xe11f, 0x0400, 0xc276,  0x0050, 0xe11f, 0x0400,
0xc076 },
{   0,    5,  0x0550, 0xb308, 0x1000, 0x892a,  0x0550, 0xb308, 0x1000,
0x882a },
{   0,    6,  0x0010, 0x0000, 0x22c9, 0x0522,  0x0020, 0x0000, 0x22c9,
0x0522 },
{   0,    7,  0x0010, 0x0000, 0x22c9, 0x0522,  0x0010, 0x0000, 0x22c9,
0x0522 },
{   0,    8,  0xfd20, 0x6000, 0x2069, 0x3a48,  0x0010, 0x6000, 0x2069,
0x3a48 },
{   0,    9,  0x0010, 0x0000, 0x22c9, 0xb12c,  0x0020, 0x0000, 0x22c9,
0xb12c },
{   0,   10,  0x0520, 0x6000, 0x2069, 0x3a48,  0x0010, 0x6000, 0x2069,
0x3a48 },
{   0,   11,  0x0010, 0x0000, 0x21c9, 0x1574,  0x0020, 0x0000, 0x21c9,
0x1574 },
{   0,   12,  0x0010, 0x0000, 0x21c9, 0xee7f,  0x0020, 0x0000, 0x21c9,
0xee7f },
{   0,   13,  0x0050, 0xe11f, 0x0200, 0xa2ba,  0x0050, 0xe11f, 0x0200,
0xa0ba },
{   0,   14,  0x0050, 0xe11f, 0x0200, 0xa1ba,  0x0050, 0xe11f, 0x0200,
0xa0ba },
Boot record, EDT, and emx.info from B2 (the failing node):
----------------------------------------------------------
Aug 27 15:11:35 b2 vmunix: emx0 at pci1 slot 5
Aug 27 15:11:35 b2 vmunix: KGPSA-BC : Driver Rev 1.21 : F/W Rev 3.03A1(1.31)
: wwn 1000-0000-c922-2cb1
Aug 27 15:11:35 b2 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x31000
assigned tgt id 11 - out of range for CAM
Aug 27 15:11:35 b2 vmunix: emx0: emx_assign_fcp_id: nport at DID 0x21100
assigned tgt id 12 - out of range for CAM
Aug 27 15:11:35 b2 vmunix: scsi16 at emx0 slot 0
Aug 27 15:11:35 b2 vmunix: rzb128 at scsi16 target 0 lun 1 (LID=0) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:35 b2 vmunix: rzc128 at scsi16 target 0 lun 2 (LID=1) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:35 b2 vmunix: rzb132 at scsi16 target 4 lun 1 (LID=2) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:35 b2 vmunix: rzc132 at scsi16 target 4 lun 2 (LID=3) (DEC
HSG80            V84F) (Wide16)
Aug 27 15:11:35 b2 vmunix: Type 0xc at scsi16 target 5 lun 0 (LID=4) (COMPAQ
DATA ROUTER      1170)
Aug 27 15:11:35 b2 vmunix: changer at scsi16 target 5 lun 1 (LID=5) (DEC
TL810    (C) DEC 2.31)
Aug 27 15:11:35 b2 vmunix: tzc133 at scsi16 target 5 lun 2 (LID=6) (DEC
TZ89     (C) DEC 2150) (Wide16)
Aug 27 15:11:35 b2 vmunix: tzd133 at scsi16 target 5 lun 3 (LID=7) (DEC
TZ89     (C) DEC 2150) (Wide16)
CAM Equipment Device Table (EDT) Information:
    Device: TLZ10      Bus: 1, Target: 5, Lun: 0, Type: Sequential Access
    Device: CDR-8435   Bus: 2, Target: 0, Lun: 0, Type: Read-Only Direct
Access
    Device: HSG80      Bus: 16, Target: 0, Lun: 1, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 0, Lun: 2, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 4, Lun: 1, Type: Direct Access
    Device: HSG80      Bus: 16, Target: 4, Lun: 2, Type: Direct Access
    Device: DATA ROUTER Bus: 16, Target: 5, Lun: 0, Type: Array Controller
    Device: TL810      Bus: 16, Target: 5, Lun: 1, Type: Medium Changer
    Device: TZ89       Bus: 16, Target: 5, Lun: 2, Type: Sequential Access
    Device: TZ89       Bus: 16, Target: 5, Lun: 3, Type: Sequential Access
    Device: KGPSA-BC   Bus: 16, Target: 6, Lun: 0, Type: Processor
  emx? tgtid  FC Port Name                     FC Node Name
{   0,    0,  0x0050, 0xe11f, 0x0400, 0xc176,  0x0050, 0xe11f, 0x0400,
0xc076 },
{   0,    1,  0xfd20, 0x6000, 0x2069, 0xe826,  0x0010, 0x6000, 0x2069,
0xe826 },
{   0,    2,  0x0010, 0x0000, 0x22c9, 0xb12c,  0x0010, 0x0000, 0x22c9,
0xb12c },
{   0,    3,  0x0420, 0x6000, 0x2069, 0xe826,  0x0010, 0x6000, 0x2069,
0xe826 },
{   0,    4,  0x0050, 0xe11f, 0x0400, 0xc276,  0x0050, 0xe11f, 0x0400,
0xc076 },
{   0,    5,  0x0550, 0xb308, 0x1000, 0x892a,  0x0550, 0xb308, 0x1000,
0x882a },
{   0,    6,  0x0010, 0x0000, 0x22c9, 0x0522,  0x0020, 0x0000, 0x22c9,
0x0522 },
{   0,    7,  0x0010, 0x0000, 0x22c9, 0x0522,  0x0010, 0x0000, 0x22c9,
0x0522 },
{   0,    8,  0xfd20, 0x6000, 0x2069, 0x3a48,  0x0010, 0x6000, 0x2069,
0x3a48 },
{   0,    9,  0x0010, 0x0000, 0x22c9, 0xb12c,  0x0020, 0x0000, 0x22c9,
0xb12c },
{   0,   10,  0x0520, 0x6000, 0x2069, 0x3a48,  0x0010, 0x6000, 0x2069,
0x3a48 },
{   0,   11,  0x0050, 0xe11f, 0x0200, 0xa1ba,  0x0050, 0xe11f, 0x0200,
0xa0ba },
{   0,   12,  0x0050, 0xe11f, 0x0200, 0xa2ba,  0x0050, 0xe11f, 0x0200,
0xa0ba },
{   0,   13,  0x0520, 0x6000, 0x2069, 0xe826,  0x0010, 0x6000, 0x2069,
0xe826 },
{   0,   14,  0x0010, 0x0000, 0x21c9, 0x1574,  0x0020, 0x0000, 0x21c9,
0x1574 },
{   0,   15,  0x0010, 0x0000, 0x21c9, 0xee7f,  0x0020, 0x0000, 0x21c9,
0xee7f },

Received on Wed Aug 29 2001 - 11:44:28 NZST