Thanks to the many people who responded to my question. The concensus seems to
be RAID-0 gives the best performance but no data redundancy. The next best
performance option which also provides redundancy is RAID-0+1. RAID-5 is not a
good performer if you're application is doing a higher percentage of writes
compared to reads and should be used as a last resort in that case. Most people
have configured a separate logical RAID (0+1 or 1) drive for their system drive
for "/", "/usr",swap, "/tmp" and then setup other RAID drives for the
non-system stuff so it's not recommended to stuff everything (system and
user-data) onto one big logical drive. The bottom line seems to be that the
type of RAID to use depends on the I/O characteristics of your application as
well as the amount of $$$ you have available to spend for disk.
Here's my original post:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I'm in the process of installing a DS20 (2CPU 2GB) server that will
be running DEC Unix 4.0E. This system has 7 x 9.1GB storageworks
drives in total with 4 drives on channel 1 and 3 drives on channel 2
of a KZPAC controller.
I'm trying to decide which is the best RAID configuration for optimal
system performance. Initially I was planning to create a RAID-1 set
(utilizing 2 drives) for my system disk thinking that a dedicated
mirror set would provide quickest access for high priority system
tasks such as swapping and paging etc. I was then going to put the
remaining 5 drives in a RAID-5 set to be used for user data and apps.
Having second thoughts on this approach, I'm now wondering if I
could achieve better overall performance if I configure all my disks
into a single RAID-5 array and then create partitions for "/",
"/usr", swap, "/tmp", "/usr1/data", and "/usr1/apps" within this
logical drive. This way I gain the performance benefits of RAID-5
striping across all 7 disks but will still incur some overhead in
maintaining the parity stripes. Would this second scenario be a
approach for overall performance??
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here are the replies I got:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
From:
"Degerness, Mandell ISTA:EX" <Mandell.Degerness_at_gems2.gov.bc.ca> |
Block address
Subject:
RE: Best RAID level for system disk???
Just my 2 cents...
I would generally want the system disk isolated from the rest of the
systemas much as possible and to have maximum redundancy. To this end, I
wouldwant to use the RAID1 for the system (/,/usr,swap) and RAID5 for the rest.
I might even reconsider the RAID5 depending on performance and storage
requirements. (Some tests I have seen show mirrors outperforming RAID5
forcertain types of applications.)
------------------------------
Chris,
If you can get an 8th drive, you'll achieve the best perfomance from a
0+1 RAID set, but wait, I'm not sure a KZPAC can handle 0+1. I have a third
party MTI box that I'm doing it on, and it's fast. Stripped mirrors, you can't
beat it for speed.
Jim Fitzmaurice
------------------------------
Reply-to:
Chuck.Taylor_at_Vishay.com
Hi Chris,
I find your question very interesting. You must already know that RAID is not
promoted as a means of enhancing performance and any sort of RAID may cause
additional overhead that you would not incur elsewise. Nevertheless,I believe
that it is possible to enhance not only availability but also performance. The
"book" says that RAID 0+1 is the best of all worlds right now. That is, you
create RAID 1 (mirror) sets and then stripe (RAID 0) the mirror sets together.
I have used this configuration a lot. Especially with databases.
Whether or not you could achieve a performance enhancement or not by going with
all RAID 5 would depend on a lot of factors but the chief among them would be
write verses read activity. If you do mostly read activity then RAID 5
provides a high level of performance. When writing to a RAID 5 array
your system will always do two writes every time it does a write. With
that in mind, you might not want to have your swap space on a RAID 5 array -
albeit a seven member R5 array. You will increase the read performance
over a single spindle a lot but you may not increase the write performance at
the same time.
With a RAID 0+1 or RAID 10 array on the other hand writes are in parallel
and reads are asynchronous. Thus, write performance is basically the same
as not doing RAID at all and read performance can be much better. You can
also create three member mirror sets which will allow you break out one
member to do backups from or to use as an on-line backup (by only being in
the mirror set for the period of time needed to synch the data on all
members of the mirror set).
The reason I find you question interesting is that to find the answer to
it you would have to do some serious math and you would have to also
understand the nature of the workload that would be using the disks. If you
knew all of that well enough to do the calculations then you could probably
answer the question.
I would suggest that you visit the RAID Advisory Boards website and see
what they might have to say on the subject.
I would also be very interested in seeing what other responses you get to
your questions. Please provide either a summary or reply back to me when
you get a chance.
I hope this helped...
Chuck Taylor
Senior Infrastructure Developer
Vishay - Siliconix
2201 Laurelwood Road
Santa Clara, CA 95054-1595
(408) 970-5381 Phone
(408) 567-8967 Fax
chuck.taylor_at_vishay.com
-----------------------------------
Subject:
Re: Best RAID level for system disk???
Date:
Wed, 20 Oct 99 16:15:26 -0600
From:
alan_at_nabeth.cxo.dec.com | Block address
It depends on what is on the system disk.
Root - The odds are good that the most frequently used files
in the root will end up being cache anyway. Without any luck
the day to day performance of this partition won't be relevant.
Swap - If the performance of I/O to the page/swap space becomes
a real issue, you need more memory. If that isn't an option
you should spread the I/O out over more devices and controllers.
/usr - If your work-load is fairly restricted (everybody running
the same thing), you may get the cache benefit as the root and
I/O performance won't be an issue. When /var isn't part of
/usr it tends to become read-mostly.
/var - This is probably write-mostly and you can't avoid some
I/O load.
/tmp and /var/tmp - I once collected I/O traces on an ULTRIX
system and found that 98%+ of the I/O to /tmp was writes. This
is to be expected, since it tends to be small temporary files
that will still be cache when read back. If I/O here is an
issue and you have spare memory, think about using a memory
file system (% man 8 mfs). The temporary files in /var may
expect to be more persistent, so I wouldn't use a memory
file system there. But, as noted earlier, /var is write-
mostly anyway.
Application - Does the application run a single monolithic
copy of itself or have shell application that loads and
runs smaller programs and scripts as needed? In the single
monolithic program design once one person is running the
thing, starting up another copy takes little more than
creating the virtual memory and setting the access counts
for the shared text and data; not much I/O. Too big a
monolith and unused parts may be paged out, which means
they need to be paged back in as needed.
A shell application may have to run lots of little programs,
but they may also fit in the buffer cache if frequently used
enough.
Application Data - Is it a small read-mostly data base, small
read-write database or a small write-mostly database? Writes
are what hurt, since "small" may fit in the cache for reading.
The thing that hurts the most in disk performance is having
to do any at all. The more data that's in the cache, the
less that has to be read. Tru64 UNIX uses a unified buffer
cache that makes all of free memory available to the cache.
If the application uses lots of memory, this takes away what
the cache can use. Sequential searches of large files tends
to take away from what programs can have (read-ahead).
Writing always takes I/O if you want the data to be persistent.
Unless you're doing large sequential writes, the thing that
typically limits write performance is seeks. If you have
write traffic, you can try one of two things:
o Cluster all the write traffic to a narrow range of the
device. While this is most applicable to single devices
it does apply to arrays as well. Keeping the write close
together reduces the likely seek distances. If the buffer
cache is doing its job, reads will be less common. However,
if the data to be read is farther away, it will mean longer
seeks when you do need to read.
o Isolate the write traffic of each file system to an independent
device. In an array, you have as many independent devices as
you have members (members - 1 for RAID-4 or RAID-3). If you
have multiple file systems on a single partitioned array,
the load may evenly balance, getting the desired affect.
If performance is the only concern, Striping is better than
mirroring or RAID-5 because it doesn't waste capacity on
redundancy. If some redundancy is needed, RAID-5 is good
in a read-mostly environment. Small random writes can hurt
since they probably have to touch more members to complete
an I/O. In a random write-intensive I/O load, mirroring is
probably better than RAID-5. For sequential writes, a smart
controller may be able to pretend that RAID-5 is really
RAID-3 and perform appropriately. If cost isn't an issue
RAID 0+1 because it offers the benefits of striping and
mirroring.
And, after all of this, I haven't answered what I think the
core question is: Single big partition RAID or separate devices.
With 7 disks spread over three channels, there are going to be
times when one disk may be able to transfer data, but can't
because another disk on the bus already is. Multiple disks
on a single bus do allow seeking in parallel though.
I don't know that there's a clear advantage of 7 disks in a
single RAID over a couple or three disks in two arrays since
they have to share the same three busses anyway. If you're
I/O bound because of seeking, whichever reduces the average
seek distance is the one that will win. If you're I/O bound
for data rate and the load is balanced across the three
(or two) channels, the controller is the bottleneck and
there's not much you can do except get a 2nd or 3rd controller.
If you're not I/O bound, as long as you don't make it worse
by reorganizing it doesn't matter. If you have the leisure
to experiment with the new system, benchmark it both ways
and see if one is better. If the difference is marginal,
use the method that is easier to manage.
-----------------------------------
From:
LARRY.CLEGG_at_LPL.COM | Block address
Subject:
Re:Best RAID level for system disk???
To:
sirhc08_at_yahoo.com
DO NOT DO THIS!!!!! Go with your original plan of a mirrored system disk. If
you've got the space then I'd recommend that you use RAID 0+1 (striped mirror
sets) for the remaining partitions - use RAID-5 only as a last resort. You'll
get much better performance with RAID 0+1.
I'd also be using Unix V4.0F over V4.0E. 'F' is much cleaner. I'm already
running V5.0 for two DNS servers and I really like it.
Hope this helps,
Larry
-----------------------------------
For some of the critical servers we have, we will mirror the system
disk, then put data on RAID5 disks. Our reasoning? If any system
partition fails, you are still running. You can replace the disk and
still stay running. (Possibly a reboot may be required.) Sometimes, we
won't mirror, just put in an identical disk, then we simply make the 2nd
disk bootable, and dump the /, /usr and /var partitions to the backup
disk. Any data files that, say, your applications and databases would
use would still reside on the RAID5 partition. This would allow you to
have a failed system disk and still come up. Also, if you had a RAID
problem, you could still bring the system up without the RAID software
or hardware.
It's all a trade-off on what you are willing to work with. If you have
a very serious failure, having all the disk space in RAID5 could take a
while to recover from. By having part of it mirrored, and the other part
in RAID5, I think you would reduce the downtime from a serious failure,
since you have isolated different types of data areas. Remember that
mirrored writes take a little longer than a single write, however
mirrored reads read twice as fast. That usually will work well for your
/, /usr and /var partitions. Other than /var, they are pretty static,
and most of the stuff in /var is usually small.
We had a mirrored system that somehow managed to get the / partition
corrupted. Well, we had 2 copies of an unusable root partition. Fat lot
of good the mirroring does you then. You have to rebuild the system and
the mirror. Fortunately, we had our data on some RAID5 partitions, so
that was still good. It took us a while, but we got the system back,
with all the RAID5 data still usable.
Just my opinion.
Bryan
--
Bryan Bahnmiller - Lead Systems Administrator
Raytheon Systems Company
-----------------------------------
From:
"Anthony A. D. Talltree" <aad_at_lovecraft.talltree.net> | Block address
To:
Chris Los <sirhc08_at_yahoo.com>
Subject:
Re: Best RAID level for system disk???
Disks are cheap. I prefer mirroring instead of RAID 5. I'm not sure if
you're talking about doing RAID 5 in hardware or in software, but I can
assure you that you would not be pleased with the performance of the
latter.
-----------------------------------
From:
John Clifton <John.Clifton_at_keltecprogress.co.uk> | Block address
Subject:
RE: Best RAID level for system disk???
Date:
Thu, 21 Oct 1999 08:46:35 +0100
I would say your first scenario is better. Mirror the system disk for
security rather than performance. Swap space should be split over volumes
(for best performance), so I would put half on the mirrored pair and the
other half in the RAID5 set.
John
-----------------------------------
From:
"Andrew Tolmie" <Andrew.Tolmie_at_carltontv.co.uk> | Block address
To:
<sirhc08_at_yahoo.com>
Subject:
Re: Best RAID level for system disk???
Chris,
I have found that using the first method to be a better performer.......
2 drives configured to RAID 1 and the rest configured at RAID 5 for data and
apps.
However it must be noted that I use this configuration for Oracle DBs and
Oracle Application Server.
Regards
Andrew Tolmie
__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com
Received on Tue Oct 26 1999 - 14:32:42 NZDT