SUMMARY: Benchmarks

From: Bart.Rousseau <roussea_at_uia.ua.ac.be>
Date: Tue, 16 Mar 1999 14:46:05 +0100 (MET)

Hello,

The question was how one should compare the performance/price
for different machines or put another way, should one buy
one alpha-machine or a few intel-machines?

>From the number of responses I got it is clear that this
question is of interest to many people. This is why I took
the time to compose this rather lengthy (sorry for that)
summary...

The most general conclusion from the responses is: the answer
to this question is far from trivial.

The best way to test the relative performance of different
machines is to run your programs on all of them and see
which performs best. The problem is that in doing this you
are not only comparing different machines but at the same
time different operating systems, compilers and often even
different versions of the program. On the other hand
I don't think it is important whether increased performance
is due to better hardware or due to a better compiler, its
the increased performance that counts.
This approach is of course only valid if you will run only
a limited number of programs (which is true in our case).

In connection to the question if spec-numbers are reliable
to compare the performance of different machines, the general
feeling was to be very carefull when using these numbers.
For example someone noticed that the relative performance
of different machines was in better agreement with relative
specInt number than with relative specFp numbers although
the program involved floating point operations.
Another problem associated with this is how well the program
is optimised for a given machine. Since our main interest
is quantum chemistry and not computer science we are
not going to spend lots of time trying to optimise our
code for a specific machine (we have the time nor the
knowledge/interest to do this).

Concerning the OS the general feeling was that NT is indeed
slower than UNIX, but that in general the compiler is more
important than the OS.

After sending this message we also contacted our Digital/Compaq
dealer and found out that the price difference between
an Alpha-sytem and an Intel-system is not nearly as large
as we thought it was (the price of an Alpha has dropped
by almost a factor of ten since the time we have bought ours
(about two years ago)).

Another comment appearing in many responses was that if
you run multiple small jobs you could buy a few PC's to do
the job, whereas if you run one large job an Alpha would
be better.

Someone also suggested that we might buy two Alpha EV5
instead of one Alpha EV6. Since we are doing our
calculations in parallel (using pvm) this is definitly
something we should look into.

Keeping all of the above in mind I think we will either
buy one Alpha EV6 or two Alpha EV5 machines.

Finally I would like to mention that the majority of the
messages said something like 'go for the Alpha',
but maybe this is due to the list in which I asked
this question? ;-)

Thanks to:

   Salvador Ramirez
   Joe Fletcher
   John Francini
   Kevin Reardon
   Matt Harrington
   Reginald Beardsley
   Russel Morison
   Peter Stern
   Paul Casteels
   Alan Rollow


Thanks again,
Bart.
Received on Tue Mar 16 1999 - 13:48:59 NZDT

This archive was generated by hypermail 2.4.0 : Wed Nov 08 2023 - 11:53:39 NZDT