Hello managers,
My original question was regarding high volume tape backup. I
categories the replies into four category. The replies that can
not be fit into first four category, are in fifth.
I have not reach to any conclusion, but I like to send summary
for the people curious to see the replies. If you like to get
all the replies as it is, please drop me a mail.
Rajesh Gupta
General Sciences Corporation
Laurel, MD 20707
My many thanks to following people for their quick response:
Kurt Watkins, watkins_at_howie.swmed.edu
arun sanghvi, sanghvi_at_proto.wilm.ge.com
Rick Beebe, BEEBE_at_BIOMED.MED.YALE.EDU
Wayne Flowers, wayne_at_aprf.arl.mil
Les Troyer, t2078_at_dcs2.nfuel.com
gachamb_at_milp.jsc.n
Eugene Chu, chu_at_musp0.Jpl.Nasa.Gov
dick seymour, SEYMOUR_at_LEPTON.NPL
Adrian Ho, <adrianho_at_nii.ncb.gov.sg>
Hellebo Knut, Knut.Hellebo_at_nho.hydro.com
ELEX.Kushner.Micha, micha.kushner_at_elex.co.il
Andrew R. Tefft, teffta_at_crypt.erie.ge.com
S. D. Raffensberge, sdr_at_rdga3.att.com
James Anderson, jaa_at_csa.delta1.org
Bruce Warrington, bruce_at_ancc.com
Vince P. Lynott, lynottvp_at_ttown.apci.com
Sean Reifschneider, sreifsc_at_sreifsc.uswc.uswest.com
Mike Jones, jonesmd_at_aule-tek.com
Bert Robbins, brobbins_at_mako.Newbridge.COM
Bruce L. Harrell, bharrell_at_digit.com
Ross.Stocks.INSDRS, ross.stocks_at_nt.com
Sean Ward, seanw_at_amgen.com
Jochen Bern, bern_at_TI.Uni-Trier.DE
Nico Garcia, raoul_at_MIT.EDU
*********************************************************************
> (1) Effect of (n)number of tape drives on a SCSI controller.
*********************************************************************
Obviously as you add drives, the performance decreases. Things I've
read suggest that the break-even point is about when the controller is
half full. That is, utilize 3 or 4 of the 7 available SCSI addresses.
----------------
Too many tapes, as any other device, on a SCSI controller isn't good. A
fast SCSI controller should be able to handle three or four without
major problem, though.
-----------------
Only use one tape drive at a time on a controller. Tape is probably the
slowest medium on a SCSI bus. You might be able to get away with accessing
two tape drives on one controller. For speed you don't want your bcakup
device on the same SCSI contoller as the devices you are backing up.
----------------
A tape drive can't saturate a scsi controller. 7 tape drives
couldn't saturate a scsi controller. Tapes are measured in megs/min.
and scsi busses in megs/sec.
-----------------
Varies among platforms. A good rule of thumb is to not have more than 3
tape drives on a single SCSI bus. (This number, incidentally, was plucked
out of thin air -- your mileage may vary.)
-------------------
Hi Rajesh. I have not run more than two tape drives on a SCSI controller, so
I don't know how well that would work. I did, for a while, though, have
four tape drives connected to the same machine (via two controllers) doing
backups at the same time. However, I found that I was getting a lot of
errors during the backups. To see what would happen, I started two of the
backups late in the evening (11PM) and then started the other two early
in the morning (2AM). Since then, I have hardly had any backup failures.
--------------------
variety of effects, depending upon how they're set up.
if we assume 5-gig 8mm drives, they've got a 512kb internal buffer.
they accept data at 512kb/sec (0.5Mbyt/sec)
They can co-exist on a scsi bus (Fermi NAtional Lab has a number of
"tape walls"... simple CPUs feeding 7 8200-class Exabytes (plus one
hot spare). as each drive becomes available to accept a record, it
gets one... hence the total thruput approaches the theoretical sum of
the bandwidths (to the scsi bus limit)
BUT...that's for randomly distributed records... the FermiLab application
does not care -which- drive gets the next record.
Most backup applications do.
But a similar system might work for you: 7 drives on-line would let one
be rewinding, one loading and still have 5 taking data. A decent SCSI
interface should have no problem (since most are 5 Mbyte/sec).
-------------------
The number of tape drives you can put on a SCSI bus and still be able to
use effectively depends on the drives. With the Exabyte compressing tape
drives, their peak transfer rate is 5 MB/s, but sustained throughput with
no compression is 0.5 MB/s. With compression, the actual data rate into
the drive depends on how compressable the data is. In any case, a Fast
SCSI bus can support a peak transfer rate of 10 MB/s. If you assume an
average compression ratio of 2:1, that means each drive on the bus will
run at about 1 MB/s sustained. Then it looks like you can fill the bus
up with 7 drives, and probably still have bandwidth left over.
-------------------
we like to limit the number of 8-mm to 2 per controller.
--------------------
That depends on the time you have to do the backup, and the speed of
the tape drives. A fast narrow SCSI bus should be able to handle
7 DDS-2 tape drives as they're only going to run about 400K/sec, and
7 of them would be under 3MB/sec. Of course it will take just under
19 hours to do the backup with all drives running full speed.
You really need to be looking at some of the REALLY high capacity
and speed drives. Check out the comp.periphs.scsi FAQ, I believe it
lists the specs of different drives.
You probably want to look at something that can do a GB or so per minute
I'd guess.
------------------
Should be negligible (OK, OK, so I don't have first-Hand Experience ...).
A SCSI-2 (fast SCSI) Bus can do 5 MB/s and 7 Devices, a fast-wide
Bus 10 MB/s and 15 Devices, both results in 650-700 kB/s for every
Tape Drive, which is still close enough to what a DAT or Exabyte can
handle. DLT Tapes are reported to do up to 3 MB/s, but that's Vendor's
Claim and I haven't seen real World Reports on this.
------------------
you can have 7 devices on the controller max is 7ft. on the cable.
********************************************************************
> (2) Is there an option to increase the buffer on tape drive to
> improve the performance.
********************************************************************
Not that I know of, but this doesn't seem likely to be the bottleneck.
------------------
Yes, instead of using tar, cpio, or backup use a third party backup
solution or use tar and cpio with dd to increase the buffer size.
Remember the key to fast backups is to keep the tape drive light ON.
-------------------
We bought a couple drives with advanced scsi processor in them.
They have 4 simm slots and presumably came with 4 megs in them,
perhaps upgradeable to 16 megs.
------------------
Again, depends on the platform. When planning backups, assume no
effective buffering.
------------------
yes. An example is the Triplex controller for the Sony DIR family of
tape drives. It can have up to 370 Mbytes of memory for data buffering.
(the Sony drives can take up to 32 MBytes/second...)
You never mention -which- flavor of drive-style you're thinking of using.
a "drive's" memory can always be expanded by attaching the drive to a
computer with lots of memory. The Triplex controller is really a Sun
computer with extra data-dedicated memory on a private bus (since the
data in/out flow exceeds the Sun's native VME bus capacity).
--------------------
Well, the drive can only go as fast as the data can be written to tape.
My DDS-2 4GB tape drive has a MB buffer which works out to a couple of
seconds worth. I'm sure the higher capacity and higher speed drives have
larger buffers.
-------------------
We're talking about the MB/s Range here. What Hosts do you have?
How much can their System Buses deliver in the first Place?
**********************************************************************
> (3) Is it possible to backup on two or more than two tape drive
> on one controller at the same time and get the same performance
> as if the tape drives were on different controllers.
**********************************************************************
Yes and no. You can definitely utilize multiple drives on a single
controller, but you cannot avoid some performance degradation. However,
the degradation may begin at a fairly insignificant level and not become
objectionable until several drives are in use.
-----------------------
Not the same, but the tapes are not as fast as the SCSI channel so the
contention is usually manageable.
---------------------
I certainly would not use more than two tape drives on the same
controller. And I would not use them to backup any drives on that
controller. And I wouldn't backup the drives on that controller while I
was using the tape drive.
----------------------
Close to it, yes. I use two DDS drives on a single SCSI bus, and I don't
think I'll get much better performance if I had them on separate busses.
--------------------
up to the limit of the bus involved... an Exabyte will accept a burst of
data at scsi-bus speeds (since it's system memory talking to drive memory).
Then the drive's internal memory feeds it slowly to the tapeheads.
-------------------
performance goes down but how much depends on data source,network traffic,...
-------------------
Depends on the drives. Fast narrow SCSI has a throughput of 5MB/sec.
If you put two 3MB/sec drives on a single controller, you will be saturating
the bus. But you can put ~10 DDS-2 drives (depending on the data you write)
on a single SCSI II bus before you saturate it.
------------------
I would use polycenter networker for this. It comes with dec unix 3.2
**************************************************************************
> (4) What is the best hardware configuration (controllers and drives) to
> get better performance. Any advice on type of tape drives or on any
> related info will be appreciated.
*************************************************************************
I think the answer here is definitely to purchase something specifically
designed for this purpose. It will do the job more efficiently than the
workstation, and will free up some of the workstation cpu and give it a
possible performance improvement.
It's the same principle as buying an 80GB disk array versus buying 80GB
worth of SCSI drives, the controllers, cables, etc., and hanging them
all off SCSI controller(s) in the workstation. You get better
performance from the array because that is all it is designed to do, and
that is all it does. Since that is precisely what it is designed to do,
in theory it maximizes efficiency where the workstation may have to
compromise efficiency at one task in order to maximize overall
efficiency. From the size of your system, it sounds like you may have
been down this road already.
------------------------
As I said, we've gone with the Sun FSBE controllers and Exabyte drives.
The EXB-440 takes the latest drives, which I believe are the 8505.
They're 14-GB (compressed) tapes with a 1 MB/sec throughput
(compressed).
----------------------
I haven't been able to find anything faster than 5 to 10 GB per hour. So
assuming marketing literature does not lie (:-) 200 GB / 2 drives _at_ 10 GB
hr. gives you a backup window of 10 hours. This is a very long time. If
I had to back this much stuff up I would be asking stupid questions like
"How much of this data realy changes every day?" "How come we didn't put
the data that does not change on WORM devices?"
If the answer was all 200 GB changed everyday I would find a way to add a
Backup storage system. THis storage system would have a large amount of
disk space (50% or better of existing storage), Multiple high speed
connections to the existing system (2 FDDI channels min.), Lots of SCSI
controllers (1 for each 10 GB/hr tape, 1 for every 4 Hard drives). FOr
backup I would spool the data from the regular system to the backup
system. Then start backing it up on tape, erase files as they are verified.
And I would pray a lot.
----------------------
Back to the tape drives we bought. Each one is a 2-drive exabyte 8505.
The two drives can work independently, or 'cascade' mode which automatically
fills one tape and then the other, or 'striper' mode, where the asp board
writes to both tapes simultaneously. This nearly doubles our throughput.
We get an average of about 90 megs/min.
---------------------
We use Epoch (off of a Sparc) but only do a few 5-10 GB per night. Our dealer
just told us about a new Exabyte box with upto four drives. What's important
and what you may want to consider is using Fast/Wide SCSI which give 4 times
normal SCSI throughput (20:5 MB/s).
---------------------
A Fast/Wide SCSI-2 interface is probably nice. See if you can find tape
drives that match the above spec. Sorry I can't provide more details --
I haven't been following the market much...
The worst-case scenario: Buy a tape drive for each machine and do all
backups locally. It works, but it may be too expensive.
-----------------------
The worst Bottleneck that you want to avoid is Ethernet. Don't backup
remotely. This means that you need to put your Tapes where the Disks
are, and this will likely dominate all further Layout.
---------------------
Aviv now sells a Sony DIR and controller combination that works at about
2 megabytes/second. The drive is -not- cheap. But they're rock-solid.
(hmmm... my pricing information is all pre-Yen-skyrocketing... so they're
*NOT* cheap)($100k)
----------------------
We are in the process of dumping all 8mm and going with DLT (they look like
old TK50's but perform much much better). They hold ~ 20G (i think thats
uncompressed). I'm not involved in this area but the people who are using
them really love them compaired to either 8mm or 4mm.
----------------------
I would seriously look at DLT (Digital Linear Tape). These tapes are
capable of holding 18gb each and are quite fast. They were originally
manufactured by DEC, but they've sold the license to someone else now.
I think with those backup needs, you're may want to investigate jukeboxes
of some sort. There are many available--most based on DAT or 8mm drives.
Many of them hold up to 7 drives and are capable of storing >100 terabytes.
-----------------------
I just got back from a Digital seminar where the POLYCENTER Networker Save
and Restore package was briefly mentioned. Supposedly this has the
capability of doing parallel backups to the same device. Having several
machines' archives interlaced on tape sounded fishy too me, but...
----------------------
I've run 5 DAT drives on a SCSI controller, with only minimal decrease
in througput. My recommendation is NOT to buy the 8 GB dat drive that
HP offers on their 800 series and T500 series boxes, but rather to buy
the 16 GB dat drive that HP sells through another division. For some
stupid reason, one division of HP can't deal with the other, so ask
your HP var to sell you the 16GB drive, don't ask HP to include a dat
drive with the server hardware, because they won't. The 16 GB dat
drive has an internal buffer, and the hardware based compression has
given me around 75 MB/min per drive. To exceed this, you can also use
the Herstal ADIC drive jukeboxes, they'll robotically feed 4 tapes
into 4 DAT drives, then hardware compress and stripe across all 4 for
very fast backups, or you can hardware stripe across 2 drives, and
simultaneously mirror and hardware stripe to the other 2 dat drives,
which saves a lot of time if you are required to do offsite backup
copies of tapes.
-------------------
I reccommend the DEC DLT in a stacker format with either Legato or
GT Backup. We have a 5 tape DLT stacker and GT Backup. The DLT tapes,
when using compression, can hold ~20GB of data. The time to perform
backups is 1 GB in about 20 minutes. The nice thing about GT Backup
is that it allows you to do things prior to backing up individual
disks, you can run a shell script to shutdown something prior to
doing the backup.
*********************************************************************
> (5) Following are the replies I could not make them belongs to the
> above four catagory, but are very useful.
*********************************************************************
Wow. 200 GB *every day*? We're currently involved in setting up an
enterprise-wide backup solution for a large customer (I can't give
their name) who has about 600 GB on-line storage spread among Sun, HP,
SGI, and DEC Unixes, a Convex, VAX/VMS, PCs, and Macs. One of the
things we immediately decided we had to do was rotating level 0's. That
is, we realized that we'd never be able to complete a level 0 backup of
every machine on the net in one night. So we've divided them into 14
groups with a two-week cycle. Group 0 gets a level 0 on Sunday of week
1, group 1 on Monday of week one...group 7 on Sunday of week 2, group 8
on Monday of week 2... with various incrementals in between.
We're still in the implementation phase, but everything on this project
looks like it's going to work more-or-less according to plan. We went
with this hardware:
SPARCserver 1000
Exabyte EXB-120 tape robot
The SPARCserver has a fair bit of fault-tolerance. We configured it
with two CPU boards, 1 CPU/board, so that if a CPU or board fails the
machine will reconfigure itself around the failing component. The
EXB-120 has 4 8mm drives, and can run them in parallel for a net
throughput of between 2 and 4 MB/sec. This is about as fast as you're
going to be able to get data (a) down the SCSI chain and (b) down the
ethernet.
We're using the SM-arch backup product from Software Moguls. It's a
full networked client/server backup package with a GUI interface for
scheduling and restores that runs on a wide range of clients. It will
handle most standard tape robots. The EXB-120 we're using is in the
process of being replaced. I'd look into an EXB-440, which just came
out.
------------------------
You ask a lot of good questions. However, you didn't mention
if some or all of these 200MB will be backed up over a network.
Not knowing exactly what you have, I can offer the following
suggestions based on the 200MB figure.
1. Distribute tape drives (or stackers) among the machines so
as to eliminate as much network traffic as possible. Backing
up over a net slows things down considerably and the occasional
users who try to work while backups occur really hate what it
does to their performance.
2. Where possible, attach a tape drive to the SCSI bus con-
taining the disks it will back up. Doing so will allow more
parallelism without sacrificing performance.
Of course, doing these things will make backup management a
nightmare. However, getting that much data onto tape in a reasonable
amount of time (less than a weekend) will be difficult otherwise.
-----------------------
Consider Legato NetWorker or DECnsr (DECnsr is DEC's version of NetWorker,
server versions for ULTRIX, ALPHA, VAX). These two backupsystems have a
kind of software multiplexer that ensures high transfer rates to all
tapedrives even if sitting on the same controller.
-----------------------
Ouch! We currently back up 40GB in our cluster, and it already takes a
helluva long time (16+ hrs on two tape drives).
One thing I can say up front: If you're doing network backups over
standard Ethernet, it'll take at least *57 hrs* to backup 200GB to a
single server (assuming a best case throughput of 1MB/s). You're
obviously not gonna be able to do this daily.
First of all, consider doing incrementals instead -- they typically take
up a lot less bandwidth. You should still do full backups at regular
intervals, but then you have the option of partitioning your network into
backup groups, only one of which is doing full backups on a particular day.
Also, shoot for completing the backups in under 8 hours if possible.
Again, use the 1MB/s Ethernet throughput figure to do your calculations.
The 8 hour time limit is about what most organizations can sustain, and it
gives you a sizable time buffer (ie. the rest of the day) to redo a backup
gone sour.
-----------------------
It has been a long time since I have had to deal with high speed high volume
backups, and this was data acquisition more than backup. In any case that
200G/day seems a lot for scsi- when I was fussing with the problem we used
high speed 9tr drives with vacuum tensioners. I forget the interface, wasn't
scsi, might have been sti (standard tape interface).
Hopefully somebody with more recent mainframe experience than mine will
remember details better! You might also contact some of the major tape
manufacturers like Ampex.
----------------------
We are working very hard to find the "perfect" backup solution as
well. I will be very interested in your findings. I afraid we have not
found the right combination yet but here are few things we found out.
We have tried
4mm STK/Logo datawheel
8mm STK
DLT (DEC repackaged by Box Hill)
All of these were teseted on an IBM RS6000 running
several backup packages. Networker, Open Vision and
one more that escapes me.
Our goal
We were trying to backup systems running hp-ux, AIX and NT
to the same server. Approximatly 100G a night.
Results
Networker software gave us the ability but the hardware
gave us reliability problems and the network limited our speed.
The 4mm was the most flexible and when the drives were
locally attached the speed was good 700K to 900K per second
This was largly do to Networker being able to run several
file systems in parallel. Over the network though it
would only ever open up one filesystem per server and the
speed went to a steady 400K. Also we were trying to use a
full barcode functionality and we had several problems
with the robotic interface. Communications to the robot
would hang and require a complete restart of the backup
software. Open Vision Netbackup has several good features
but they have no NT interface yet. With Open Vision you
can keep a central backup server but remote the drives.
By doing this you can pick the large systems and use local
drives or stackers on them at the same time you run
backups for smaller systems over the wire back to drives
on your backup server. Speed is much less of a problem
and you have a central database of backup volumes.
---------------------------
All of your questions and concerns can be solved by useing Legato's NetWorker.
The softwre supports multiple drives. With this much data to be backed up
you should try and dedicate a machine to backup all of your data. Also,
I suggest looking into tape library units. Exybte makes a tape library that
has four tape drives which can be written at same time. You should see about
500 to 700kb/s on each drive.
There is not much you can do about tape buffer. Legato also has a board
which is basically a buffer for tape drives. Call presto serve card.
Call (415) 812-6100 and ask there support department to help you!
------------------------
I couldn't tell from your message if the 200GB was the size of the file
system to be backed up or the expected size of the dump files.
You also did not say what type tapes you wanted to use. I think you'll
want to consider no less than 8mm. I don't know if there are higher
density media available or not. I think the technology can now squeeze
a maximum of 10GB onto an 8mm tape. It's possibly even greater than
that by now.
If 200GB is the expected total size of the dump files, you probably
should look into some sort of tape carousel or silo because you'll be
needing 20 tapes for a day's dumps. If you are not taking advantage of
incremental dumps, I would definitely look into that to reduce the
amount of tapes you'll need.
-----------------------
200 Gig/day? *Sheesh*. You need at least 3 auto-loading 4mm DAT drives.
You also need to think about your bandwidth:
10 MHz ethernet
~1 Mbyte/second full load
100 kbyte/second typical before loading down other traffic
100,000 seconds/day
= 10,000,000,000 bytes/day
= 10 Gig/day
That means you need to be distributed over about 10, maybe 20 subnets
to prevent bogging down your network. I'd suggest you rethink your
backup system to reduce this considerably.
Received on Fri May 19 1995 - 14:54:54 NZST