Thanks to:
alan_at_nabeth.cxo.dec.com
Tom Webster
Harald Lundberg
Eric Z. Ayers
See the end of this message for the original post, and pertinent replies.
Since I am operating the Alpha in a relatively small (25 user) environment,
I've decided not to implement LSM- it would only add unneeded complexity.
The feature that I really want is the ability to extend (and perhaps
rarely, condense) filesystems without having to dump everything to tape and
restore it. I work in a sw development environment, and it really helps to
have that flexibility as projects come and go. AdvFS would appear to fit
the bill.
I do have another general question. I have 2 4GB RZ-29s and one 2GB RZ28.
Does anyone have any recommendations regarding how the domains should be
set up on these disks? In one of DEC's docs, it says "for best performance,
avoid splitting a disk between two file domains." However, the FIS came
with root in one domain and /usr in another on the same disk. Is this a
good idea? Or should I just create one domain on each of the remaining disks?
============================
Original Post-
I just received an AlphaServer 1000A, and I am in the process of setting it
up. I've RTFM (sparse as those M's are), and I'm trying to get a grip on
Advanced File System and Logical Storage Manager.
First, what's the relationship between AdvFS and LSM? Specifically, what
does each provide me in terms of disk spanning, extending file systems,
etc. I have used Logical Volume Manager on HP-UX in the past. How does LSM
compare?
Second, what gotchas do I need to watch out for in implementing AdvFS/LSM?
More to the point, I can read Digital's rhapsodic reasons for using each,
but what are the disadvantages in implementing the packages?
=================
>From Alan:
LSM is a volume manager; it takes disk devices (partitions or whole
disks) and creates compound volumes. This volumes can be concatenated
devices, stripped devices or mirror devices. You can also mix the
different types of volumes using slices of the same devices. LSM
also supports the ability to grow or shrink a volume dynamically
but none of the underlying file systems support the feature.
AdvFS is a log based file system, which includes its own space
management features and utilities. It allows adding whole devices
as new volumes in a domain, whether they be simple disks, partition
or compound volumes created with LSM. This is dynamic.
AdvFS is very fragile in the face of I/O errors. LSM mirroring is
highly recommended. AdvFS doesn't like the file load created by
a News server (lots and lots of small files). The file system
dynamically create file metadata as needed, but the space must
be contiguous (and usually large). News quickly fragments the
space making it difficult to get. So, a common failure is that
the file system goes to create a file, can't because it can't
extend the metadata as needed and the create files, even though
there is lots of free space (just insufficient contiguous free
space).
The file system can be managed to avoid this problem, but it is
a lot of extra work. Enough that it is often easier to use UFS
instead.
==========
>From Tom Webster:
ADVFS is as easy as pie. It deals with raw disk partitions or random
sizes. Space
is added to a 'domain', which can contain one or more filesets
(filesystems). All
of the filesets in a domain draw from a common pool of free space. (In
practical
terms, most people only put one or two filesets in a doamin anyways.) If you
have the advanced tools license (in the NAS 200 PAK), you can: add
partitions,
remove partitions, defragment domains, ballance partition loads (it tries
to do it
on the fly but large file system changes may circumvent this), add
filesets, and
remove unmounted filesets -- all while users continue to work on mounted
filesets.
Now if you combine that with DECs claims that DU 4.0 will support hot-plugging
of SCSI devices (as long as the drivers are still in the kernel) in storage
works
shelves, and you have a system where you can replace a failing disk without
ever taking
the system down. Twiddle the system to get it to see the new drive, label
it, migrate
in the new partitions, migrate out the old partitions, pull the old drive,
twiddle
the system so it knows the disk is gone. I say 'twiddle' because I haven't
actually
tried it yet, so I don't know what all is involved.
Some of the other cool toys include: clone filesets and trashcans. Clone
filesets
are a funky way of thinking about journaling filesystems. Remember that in a
journaling filesystem, changes are written to a journal first. If
something should
go wrong, when the system comes back up it will complete all of the
complete journal
entries that are complete. By ignoring incomplete journal entries, you can
always
have a stable file system (some applications may not like the state a file
was left
in, but the underlying filesystem is OK). Clone filesets are a snapshot in
time,
that is maintained by making inverse jounals.
Say you wanted to keep a snapshot of a software relaease. Create a clone
fileset
and the size used by the fileset will only be the size needed to store the
deltas
from the normal fileset. The way most people use this is to backup
databases.
(1) Stop the data bases. (2) Make clone filesets of all of the filesets
the database
and indexes are on (it only takes acouple of seconds). (3) Restart the
database.
(4) Backup the clone sets at your leasure for a 'cold' backup of your
database.
Trash cans are simply areas where you can automatically migrate files that
users
delete. You WILL want to write a cron job to purge this periodically if
you enable it.
LSM is (to me) unweildy and overly complex, but it does provide support for
software
mirroring -- which makes up for a lot. If you can't afford a hardware RAID
or mirror
solution, you can use LSM to provide data mirroring. It will slow writes over
what it would take a normal filesystem, but reads are actually faster as it
uses
a round-robin system for reads. We actually use LSM to mirror the boot
disks on
our production systems (our feild service guy had told us that booting from
a SW 230 RAID array was unsupported -- wrong, but too late now). In
general you
will get better performance from hardware mirroring and RAID, and RAID is
generally
a lot cheaper solution (diskwise) if you have a lot of data. Of course you
can
run ADVFS on top of LSM, but I don't really want to discuss that.
The downside of ADVFS is it can be a bitch to recover if something goes wrong.
Since it is a journaling system, DEC has been lax in providing a good
filesystem
fixer to clean up inconsistancies. There are some undocumented (at least
to the
unwashed masses) field service tools, but nothing really great. ADVFS and
USENET
news, as well as REALLY large e-mail spools, don't get along well -- unless
you
put some thought into the setup before hand. ADVFS preallocates a chunk
of the
filesystem to store file meta-data in when a partition is added to the domain.
DEC makes certain assumptions about the number of files in the default
values that
are way too low for a news spool that has a LOT of 1-2kb files. ADVFS will
dynamically allocate more space for the metadata, but this leads to
fragmentation
and poor performance. You can manually set the amount of metadata storage
set aside
when you create a domain, if you know to do it, but you can't tune it later.
Like I said before, LSM is (to me at least) non intuitive. If you have a
requirement
to have redundancy on much more than your boot disks, you would probibly so
better
to look at hardware raid solutions.
Both system's disk/partition spanning systems have the problem that they
perform like
a simple RAID-0 setup (OK, so LSM may be able to pull off a RAID-0+1
(mirrored stripes)).
If you blow out a disk or a partition and aren't using RAID or mirroring,
you are going
to have to restore the WHOLE thing from tape.
============
>From Harald Lundberg:
LSM i.e. Logical Storage Manager is exactly what it states - Manages
Logical storage entities. It is NOT a filesystem. On top of LSM logical
volumes you can create ufs or advfs filesystems. I'd say there are 2
reasons for using LSM:
1) use the mirroring capability
2) specific need to create and remove filesystems (or raw partitions/swap)
at any time.
Basically, if you want to span multiple disks in an advfs domain (i.e. HP
volume group) then if one disk breaks your whole domain is gone. So either
use mirroring or RAID5 to get some higher data protection. Recovering a
20Gb domain with 10 filesystems just because a 2gb disk crashed isn't
nice (I personally use both a separate raid5 box and lsm in 2 different
setups).
Advfs is GREAT! Couldn't live without it anymore. LSM is a pain in the ass
to setup and administer (you get used to it, but I still think it stinks).
I'd rather recommend a RAID controller.
=========
>From Eric Z. Ayers:
You can use ADVFs to put multiple volumes into a single file domain and
add/delete storage from them at will. From my experience,
LSM has proven to be difficult to configure and maintain,
so I wouldn't use LSM if you're just going to be concatenating
several disks together to make a big filesystem .
If you want to mirror volumes, however, LSM is the (only) way to go.
If you want to do 'real' striping of disks, LSM should give you better
performance. With AdvFS, I don't believe that files are actually
striped across different volumes within a domain. Remeber that if you
do striping without mirroring, the loss of a single disk or partition
within the stripeset causes the entire domain to be lost (no matter if
you use LSM or AdvFS.)
========
Thanks again to all.
--
Casey Spangler spang_at_probita.com
Probita, Inc. 303-449-7665
"Go, and never darken my towels again!" -- Groucho Marx
Received on Fri Feb 21 1997 - 02:48:58 NZDT