Thanks to all... Original question is at the bottom of this message.
I was right it was a goofy question! However there were a number of
excellent responses (see below).
CONCENSUS,
There were two approaches to answering this issue,
1) Redesign the application and how it processes, more than likely there is
something amiss or it is not tuned properly.
2) If the application truly needs these resources find a system designed to
provide them. Don't throw money at a work-around that may or may not work.
Also, if it does work the performance may not be worth the cost.
--------------------------------------------------------------------
First off it was a DS20 (as I was correctly reminded, DS10 is a single
processor machine).
--------------------------------------------------------------------
--------------------------------------------------------------------
FROM;
Nick Hill
Rutherford Appleton Labs
ITD Systems and User Support
n.m.hill_at_rl.ac.uk 01235 445423
One point, a DS10 is a single processor machine :-)
(regarding max ram in multiprocessor environment)
On all Alphaserver multiprocessor machines each CPU can address all memory,
it is not partitioned between the CPUs. If you run one job on a
multiprocessor machine it can use all available memory that is left after
the Kernel/buffer cache etc have taken their share.
(regarding swap configuration)
Swap space on Tru64 is already RAID 0 as such in that if you have swap on
multiple disks then I believe it already stripes across these partitions.
This works well if you have your swap disks on multiple SCSI busses. You
could also use LSM to create a stripe set which would appear to Tru64 as a
single swap patition. I cannot comment on the relative merits of getting the
swapper to use multiple swap partitions versus LSM striping.
(regarding SSD swap)
I would guess Solid State swap would be faster.
--------------------------------------------------------------------
FROM:
William H. Magill Senior Systems Administrator
Information Services and Computing (ISC) University of Pennsylvania
Internet: magill_at_isc.upenn.edu magill_at_acm.org
===<Tru64 UNIX-SIG Chair>===
I believe ...3... wins.
I think that the answer to your problem (although not as cheap as a DS10)
is ultimately going to be Wildfire.
Offhand, I don't know if any of the other CPU boxes, like a DS20E, for
example, has a larger memory capacity per CPU than the 10. I'd have to look
it up.
The 10 may be cheap, but for an application that size, the 20E should be
a much much bigger win, especially if it addresses larger memory.
The 20E IS a dramatic performance jump over the 20 -- much more than you
expect!
--------------------------------------------------------------------
FROM:
Arrigo Triulzi <arrigo_at_albourne.com> - Peripatetic Wizard
Albourne Partners Ltd. - London, UK
APL Financial Services (Overseas) Ltd. - Nicosia, Cyprus
(regarding swap configuration)
Oh God, it sounds like an awful waste of good RAM... in theory it
might work but you might also hit the max. size of swap partitions,
whatever that may be. Check before throwing the money out of the
window.
|Questions;
|1) Could this work?
Yes, possibly.
|2) If it worked, would the performance be poor?
| (better than large hard drive swap?)
It most definitely won't be remotely as good as proper RAM, you will
have I/O constraints as the swap RAID0 will still opearate through
your FWD controller, i.e. you are talking Mb/s bandwidth compared to
Gb/s memory bandwidth. That is already a factor 1000. You are going
to gain from the disk latency and transfer rate which should be able
to saturate the bus but that is pretty much it.
|3) What other solutions may exist.
I am surprised that you need that much RAM... you could always see if
a Cray T90 could do the trick although I don't know if they are still
fashionable these days. Clustering over memorychannel might do the
trick, perhaps re-writing the application as a set of tightly
interconnected processes to run over a cluster (MPI perhaps?). Also,
how about a Tandem? They definitely know how to deal with big data.
Before throwing money at the problem try looking at both the code
and/or data set. Then consider calling NASA who also have a few Tb in
data coming from all the satellites and stuff, they might point you in
the right direction if someone there is polite enough to take your
call.
Arrigo
--------------------------------------------------------------------
FROM:
Dr. Thomas P. Blinn
Huh? First, a DS10 is a uniprocessor system, not a multiprocessor.
Second, ALL of the "current generation" AlphaServer systems are SMP by
design, that is, EVERY processor has equal access to ALL of memory. It
*is* true that depending on the system design, there may be upper limits
on the amount of physical memory you can install in a system, and it is
also true that the DS10 is more limited than some other systems in terms
of the maximum memory you can configure.
If the process runs in a single system, then you are not running out of
virtual memory size. If the challenge is to get it to run quickly, you
have only two real alternatives:
1) Redesign the computation to improve locality of reference, that is,
to assure that memory accesses are clustered within a small set of memory
pages at any given time. Then those pages will be automatically kept in
memory by the demand paged virtual memory system, and the process will run
quickly.
2) Run the application on a system that has such a large physical memory
that all of the ill-behaved application's working set fits in memory.
NONE OF THIS IS ROCKET SCIENCE. This is the fundamentals that have been
documented repeatedly in the history of computer science.
Now, it is true that if you can use hardware features to improve disk I/O
bandwidth, you MIGHT be able to improve swap performance enough to make a
genuinely ill-behaved and poorly written application run faster. That is
remotely possible. But it's going to depend on where your bottleneck is
in the system -- solid state disks are no faster than the I/O interface
that is driving them, for example, and if they're old slow SCSI, then you
might not see any improvement over more modern real hard drives; getting
RAID striping to work with solid state disks may be unsupported, and even
if you get it to work, it may be that the RAID controller will turn out to
be the bottleneck.
There is no substitute for having a well written application. You've got
to go back to the application and understand why it needs such a large (by
normal commercial application standards) address space, and why it has a
large (by your description) working set.
As they say in stock car racing, there is no substitute for cubic inches;
the equivalent in computing is real physical memory, and unless you move
this application to a system that has adequate physical memory (this is
NOT a constraint on memory "belonging" to specific processors), it isn't
going to run really fast without careful application tuning.
Tom
--------------------------------------------------------------------
FROM;
Udo de Boer [udo.de.boer_at_live.demon.nl]
Ed,
The best thing to do is to make multiple harddisk available as swap.
This will be used by the operating system in an interleaved manner. Make
sure you use disks with a very low latency time. For your kind of disk
usage raid won't be very good. The reason for this is that the system is
waiting on the disk io and whatever raid you add will increase the
latency because it has to do calculations. Just add more disks. The use
of multiple solid state disks is possible but you will still be limited
by the scsi bus and adapter performance. Also a DS10 has only one pci
bus.
For the price of the solid state disks I would buy a bigger machine with
more memory inside and a bunch of scsi adapters with very fast
harddrives. Or I would rewrite the application.
Nowadays swapping is considered a bad thing because off the difference
in speed between main memory and disk access. In the past years disks
did not become any faster but the memory access speed did. (look at the
access speed)
regards,
Udo de Boer
--------------------------------------------------------------------
-----Original Message-----
I have a question for the list..., I hope it's not too goofy....
Let us say that there is a specific database process running on a
multiprocessor AlphaServer (DS10) (Tru64 4.0F) that requires a very large
amount of contiguous memory per CPU, more than the system board can address
(over 32 GB/CPU). Let us also say that this process cannot be broken up
into smaller pieces, such that it might as well be a single processor
machine for this one process.
In order to provide this process with the necessary contiguous memory the
swap space (hard drives) was increased and the process did run, albeit
dreadfully slow.
As I understand it, in a multiprocessor or clustered environment each CPU is
allotted a fixed (maximum) amount of memory and one cannot "borrow" /
address memory from other CPU's above a maximum limit.
If so...
In order to increase speed and available contiguous memory could a RAID 0
volume be created using all Solid State Disks and declared as swap, while
simultaneously decreasing the maximum amount of memory allotted to this
specific process in order to force it into using the newly created
"FastSwap"?
Questions;
1) Could this work?
2) If it worked, would the performance be poor?
(better than large hard drive swap?)
3) What other solutions may exist.
Thanks in advance for any responses, I will summarize.
Ed Notari
Received on Wed Feb 23 2000 - 18:31:32 NZDT