I would like to thank everyone for their responses.
The responses are rather lengthy but good reading
for someone (like me) who knows little about this subject.
Mandell Degerness MDEGERNESS_at_galaxy.gov.bc.c
Kurt Carlson sxkac_at_java.sois.alaska.edu
Alan Rollow alan_at_nabeth.cxo.dec.com
Frank Gallacher franx_at_oas.telstra.com.au
>From Mandell:
I do know that internally, the hsz40 has 6 scsi busses of 4
devices each. This should not make a difference as my
understanding is that you must configure all devices such that you
have a maximum of 8 "Units". These "Units" are what the HSZ40
presents to the operating system (usually each Unit is a raid set,
but you don't have to set it up this way).
You actually don't need to have the individual units configured in
your kernal configuration file (assuming 3.2C or later). You do
need to have the SCSI cards in your 2100 defined in your kernal
configuration file.
The HSZ40 has a "SCSI Target number" defined for itself on your
host SCSI bus.
The three pieces of information you need to access the "Unit" are:
1) The host SCSI bus number (call it N)
2) The SCSI Target number for the HSZ40 (call it M)
3) The "Unit" Number on the HSZ40 (mod 8) (call it O)
You can use MAKE_RAID_LUNS (in /dev) to create the device files
using the device name rzXNN where X = a if O is 0, b if O is 1,
etc and NN is (8*N + M).
For instance, we have two HSZ40's each on a different host bus (2
and 3). Each HSZ40 has 3 raid sets with Unit numbers 0-2. The
HSZ40 SCSI Target number is 1 for the one on bus 2 and 2 for the
one on bus 3.
Therefore, the device files are rza17, rzb17, rzc17 and rza26,
rzb26, and rzc26.
---------------
>From Kurt:
The host side has 4 SCSI targets, which correspond to the HSZ
'id'. Typically people use 1-4. The controller itself is
typically 7, if you run dual-reduncant the other controller
is typically 6. It looks like you use 0-3 vs. 1-4.
The hsz side support 6 individual scsi busses to the shelves.
You have 6 scsi busses feeding the HSZ which in turn feeds
1 scsi bus to the host.
>Questions:
>1) Is this right?: formula: 8 * (scsi Bus #) + (slot # on bus) = dev number
> What's the '8' represent in the formula?
'target#' not 'bus#'.
reference hsz user's guide v2.5/2.7 section 8.4.3.2.
In addition to target, you also get LUN (logical unit within target).
A vanilla scsi disk, say rz1 is host bus 0 target 1 lun 0.
If it were on the 2nd scsi controller it would be rz9 (bus#1*8 +1).
Behind an hsz you have 'units'. The same Target 1 LUN 0 device behind bus#1
would be unit d100 and still be rz9. Now, you can have hsz unit d101
which is target 1 lun 1. The "recommended" naming for this would
be rzb9. unit d102 (LUN 2), would be rzc9. etc.
Follow the recommendation.
'8' is the number of target's you can have on a bus at this time.
Since each target can have 8 LUN's, you can theoretically have
64 devices.
use: root> scu show edt
to get a map of your devices (and other scu commands for more info).
>2) For rz16 the config file is:
> device disk rz16 at scsi2 drive 128
> Is the '128' the 'dev number' or the Logical Unit Number (LUN)?
you really don't care about the 128, but what it represents is:
bus #2*64 + target #0*8 + LUN #0
note, you can theoretically 64 devices on the bus as stated above.
>3) In DecSafe am I limited to four services on each SCSI Bus?
> For example, if I have defined: rz16 disks as Service1
> rz17 disks as Service2
> rz18 disks as Service3
> rz19 disks as Service4
> How can I define a fifth service?
> Will I need another HSZ40?
I don't speak DECsafe, but to get beyond 4 with HSZ40 you need to
use non-LUN#0 units: d101 == rzb17 (on bus#2, that is)
>4) Am I limited to 8 rz28s disks in creating a logical volume using LSM?
Sorry, I don't speak HSM.
There is an undocumented /dev/MAKE_RAID_LUNS to correctly create
the device files for non-LUN#0 disks. Before that existed we wrote
our own 'c' program to do the math:
sxkac_at_glacier> uakmknod -hsz40
scsi#? 2
HSZ unit? 101
# rz17 mknod: host scsi-id 2 HSZ unit d101 target-id 1 lun 1 lu 17
#
mknod /dev/rzb17a b 8 33856 #
mknod /dev/rzb17b b 8 33857 #
mknod /dev/rzb17c b 8 33858 #
mknod /dev/rzb17d b 8 33859 #
mknod /dev/rzb17e b 8 33860 #
mknod /dev/rzb17f b 8 33861 #
mknod /dev/rzb17g b 8 33862 #
mknod /dev/rzb17h b 8 33863 #
#
mknod /dev/rrzb17a c 8 33856 #
mknod /dev/rrzb17b c 8 33857 #
mknod /dev/rrzb17c c 8 33858 #
mknod /dev/rrzb17d c 8 33859 #
mknod /dev/rrzb17e c 8 33860 #
mknod /dev/rrzb17f c 8 33861 #
mknod /dev/rrzb17g c 8 33862 #
mknod /dev/rrzb17h c 8 33863 #
i can point you to the source if you want it.
if you thought the "what's the *8" was a bad one, don't even ask about
the minor numbers (33856, etc.). good luck, kurt
--------------
>From Alan:
The HSZ40 has 6 SCSI busses to which devices can be
connected. On the host side there is a single Fast/
Wide/Differential SCSI bus. In-between the HSZ40
remaps all I/O requests from a logical unit defined
for the host side, to the appropriate device on the
other side. There needn't be any relationship between
host side logical units and controller side devices,
except as defined by the controller.
SCSI has two addressing components. On each SCSI bus
a device is given a target address. On a narrow SCSI
busses there are 8 possible addresses using value 0-7.
Address 7 is typically reserved to the controller. On
a wide SCSI bus there are 16 addresses, but Digital
UNIX currently doesn't support this. Each SCSI target
can have up to 8 logical units. What a particular logical
unit is, is entirely up to the target/controller.
The HSZ40 can present itself to the host as up to 4 SCSI
targets. Each of these targets can in-turn have up to 8
logical units. When you define units on the HSZ40 they
associated with a particular target and logical unit on
that target. For example, Unit 100 will be logcial unit
0 on target #1. Unit 405 will be logical unit 5 on target
#4.
1. This formula is sufficient when all the logical unit
numbers are zero, which gives you access to only four
devices or arrays on an HSZ40. But, the base device
is found from that. The 8 represents the fact that
Digital UNIX doesn't expect there to be more than 8
devices per bus (Part of that Wide-SCSI support problem
I mentioned earlier).
What complicates use of the HSZ40 is that it uses logical
units other than 0 for the majority of its devices, but
the operating system support for these logical units is
a bit lacking. Starting in V3.2A, all of the pieces are
in place to make it reasonably easy to use. The MAKEDEV
script can create appropriate special device file when
you use the right name. The naming convention selected
goes like this:
rz[b-h]#
The [b-h] represents the logical number of the device or
array, where b = 1, c = 2, etc. You don't need to use 'a'
because it is logical unit 0 and can use the base device
name. The '#' is the device number found from bus*8+target.
So, using my earlier examples, and assuming that those devices
were are bus 1, the base device number for Unit 100 would
be 9 (8 * bus1) + 1. For Unit 400 it will be 12 (8 * bus1) + 4.
Since Unit 100 is logical unit zero, the base name is enough
for MAKEDEV:
rz9
for the other, you need to allow for the logical unit:
rzg12
To create special files for both:
./MAKEDEV rz9 rzg12
2. For most purposes, you don't need configuration file
entries for devices after about V3.0. It is a complication
you don't need to worry about.
3. I don't know anything about DECsafe. At a guess, once
you have used up the logical unit 0 names, start using
logical unit 1 and "rzb":
rzb16 disks as Service...
If that doesn't work, call the CSC and complain about the
lousy DECsafe documentation that doesn't explain how to
use our RAID Arrays. Of course the documentation might,
but I've never read it.
4. I don't think so, but whether you want to depends on what
you're building. If you want a stripe set, realize that
each additional member increases the probability that one
member will fail taking the whole stripe set with it. A
concatenated volume is in much the shape.
re: Your configuration.
I would spread out the disks across the busses, instead of
down them. Only once device at a time can transfer data
on a SCSI bus. One of the reasons for having six busses
is to allow up to six transfers to run in parallel. The
other is to allow connecting lots of devices.
----------------
>From Frank:
My site has an AlphaServer 2100 4/725 RM Series running Digital UNIX 3.2c,
with RAIDsets on a Digital StorageWorks HSZ40 Controller in a
dual-redundant configuration.
Hope this helps...
> 1) Is this right?: formula: 8 * (scsi Bus #) + (slot # on bus) = dev number
> What's the '8' represent in the formula?
Correct, reading from my manual, for calculating the number in the
"special device" file:
8 is the number after 7, but is actually the number of SCSI targets
you can have on a narrow SCSI buss, in the config you describe typically
6 and 7 would be used by the controllers on your 2100s (KZPSAs or something
else), lower Targer numbers in use by your HSZs.. (In the "Y" cable Tri-link
Connector Config.)
> 2) For rz16 the config file is:
> device disk rz16 at scsi2 drive 128
> Is the '128' the 'dev number' or the Logical Unit Number (LUN)?
It is the device number NOT the LUN, reading from my manual, it goes like this:
(64 * Host Side Bus # ) + (8 * Target ID) + LUN
On my system, the HSZ shows up as such:
# fgrep -i rz20 /sys/conf/TOAD
device disk rz20 at scsi2 drive 160
# file /dev/rrz20a
/dev/rrz20a: character special (8/36864) SCSI #2 HSZ40 disk #160 (SCSI ID
#4)
Please dont ask me how you calculate "(8/36864)"
BTW, I found a script called "/dev/MAKE_RAID_LUNS" that was kinda invaluable
for running /usr/sbin/mknod to create the correct "special" device files. Check
it out..
> 3) In DecSafe am I limited to four services on each SCSI Bus?
> For example, if I have defined: rz16 disks as Service1
> rz17 disks as Service2
> rz18 disks as Service3
> rz19 disks as Service4
> How can I define a fifth service?
> Will I need another HSZ40?
My site actually aborted the mission on "DEC*Safe Available Server Environment
(ASE)"; my boss decided having a "Hot Spare" CPU was an overkill..
I know you have to be *spot* *on* with your controller and disk firmwares.
> 4) Am I limited to 8 rz28s disks in creating a logical volume using LSM?
I know you are limited to 36 (narrow-SCSI) units per HSZ40B,
I don't think there is a limit on how large your RAID5 sets,
Stripesets, Mirrorsets can be.. (only "backup" window limits)
(Actually, I am going on a training course on "RAID, AdvFS and LSM" next week,
and may be able to fill you in on the full story on this one then..)
Some more gotchas:
- The batteries that were shipped with last years HSZ40s are duds, and should
be
replaced, when my trusty service engineer took them out, they said "HYUNDAI"..
- If you are going to set up RAID5 sets (ie. hot spare disks), it does not hurt
to check the failedset on a regular basis, either through the HSZterm untility:
alias hszfail '/usr/bin/hszterm -f ${HSZ} "SHOW FAILEDSET"'
Then check up on your controller to see if any disks have failed
periodically; normally the faildeset should be the null set...
or though POLYCENTER Console Manager, the HSZ40 Console ports are plugged into
a Terminal Server, PCM actually telnets to a specific tcp port (eg. 2001, 2002
etc.), and logs any events that occur; we are still experimenting with hitting
pagers when things go critical...
Remember, RAID is NOT backups, Backups GOOD, NO BACKUPS BAD!!!
Original Question:
-------------------------
Hello,
Can someone explain how SCSI the addressing works (in plain english)
on a HSZ40 controller.
I'm running with Alpha 2100s both connected to a storage array.
We are using Decsafe, LSM, and Advfs. DU 3.2C/
The way I understand it is that there are 4 SCSI slots
on the host side and 6 ports on the device side.
Questions:
1) Is this right?: formula: 8 * (scsi Bus #) + (slot # on bus) = dev number
What's the '8' represent in the formula?
2) For rz16 the config file is:
device disk rz16 at scsi2 drive 128
Is the '128' the 'dev number' or the Logical Unit Number (LUN)?
3) In DecSafe am I limited to four services on each SCSI Bus?
For example, if I have defined: rz16 disks as Service1
rz17 disks as Service2
rz18 disks as Service3
rz19 disks as Service4
How can I define a fifth service?
Will I need another HSZ40?
4) Am I limited to 8 rz28s disks in creating a logical volume using LSM?
Devices on SCSI Bus 2 (using the formula):
? scsi bus
| | slot LUN?
| | | |
rz16 = 8 * 2 + 0 = 16
rz17 = 8 * 2 + 1 = 17
rz18 = 8 * 2 + 2 = 18
rz19 = 8 * 2 + 3 = 19
Devices on SCSI Bus 3:
rz24 = 8 * 3 + 0 = 24
rz25 = 8 * 3 + 1 = 25
rz26 = 8 * 3 + 2 = 26
rz27 = 8 * 3 + 3 = 27
+------+ +------+
| 2100 | | 2100 |
+------+ +------+
| | | |
| | SCSI Bus 2 3 2 1 0 | |
| +-----------------------o-o-o-o--------- |
| 3 2 1 0 | | | | SCSI Bus 3 |
----------o-o---o-o---------)-)-)-)-------------
| | | | | | | |
+---------+ +---------+
| HSZ40 | | HSZ40 |
+---------+ +---------+
| | | | | | | | | | | |
a b c d e f a b c d e f
| | | | | | | | | | | |
| | | | | | +-+---+
===========-+-+ ====+=+==== =+=+======= ===========
r r r r r r r r r r r r r r r r r r r r r r r r
z z z z z z z z z z z z z z z z z z z z z z z z
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
| | | | | | | | | | | | | | | | | | | |
| +-+ +-+-----+-+ +-+ | +-+-+-+----+-+ +-+ +-+
r r r r r r r r
z z z z z z z z
2 2 2 2 1 1 1 1
6 5 4 7 8 6 9 7
Parts of my config file:
...
controller scsi2 at pza0 slot 0
device disk rz16 at scsi2 drive 128
device disk rz18 at scsi2 drive 144
device disk rz19 at scsi2 drive 152
...
controller scsi3 at pza1 slot 0
device disk rz24 at scsi3 drive 192
device disk rz25 at scsi3 drive 200
device disk rz26 at scsi3 drive 208
device disk rz27 at scsi3 drive 216
...
Thanks,
Lester Mayeda
Cedars-Sinai Medical Center
mayeda_at_csmc.edu
Received on Wed Dec 11 1996 - 20:00:47 NZDT