As I mentioned in the query, I'm relitivly new to this game. I had
expected that giving performance criteria for software on a new platform
should be reitivly trivial, or at the very least it should be possible
with a little bit of research.
I've since found out that benchmarking and estimation of performance is
definitly a NON-trivial area. Indeed it seems to be a sience in its own
right. One interesting site I found was the AIM one at
http:\\www.aim.com
I found the three replies VERY helpful and well constructed (which is
typicial of this list!!) so I've pretty much included them as sent
Thanks to:
Alan alan_at_nabeth.cxo.dec.com
Dave Cherkus cherkus_at_UniMaster.COM
Robert L. McMillin rlm_at_syseca-us.com
ORIGONAL QUERY:
>I'm having some problem getting some idea from a vendor of the performance of
>some database sftware I've got to get installed.
>
>We have an Alpha 3000/500s with 256M RAM. The provider (Triple G) claim that
>because they have never installed their software on this platform
>they cannot give ANY indication of expected database performance.
>
>I'm new to this game (only been on non-PC machines for 6 months) and
>just want to check:
>* Is this the crap that it sounds? Is cross platform comparison REALLY
>that difficult??
>* How would one go about making such comparisons?
>* I was only after some crude indication such as "Load Average" with
>certain amount of active users - what other performance criteria would
>you look for?
>
------------------------------------------------------------------------
-----------------------------------
REPLIES:
Alan:
I'm inclined to say your vendor is lazy, but that is too strong
and they may have good reason for not wanting to provide a
performance estimate that maybe incorrectly set your expectations.
While they may not able to give a very accurate estimate of
performance on a specific platform they should be able to make
educated guesses based on similarity of configuration with one
they have tested.
The performance of database systems involves fairly complex
interactions with system CPU performance, size of memory,
memory speed, effectiveness of CPU caches and the organization
of the I/O subsystem, which can themselves be fairly complex.
With built-in adapters a DEC 3000 Model 500 can support up to
11 SCSI disks. Using all 6 TURBOchannel adapters for add-on
SCSI you can add another 84 disks. Using the KZTSA, you
could probably have 2 HSZ40s per bus (check the SPD to see
what is supported), and six busses. With up to 42 disks per
HSZ40 you could have as many as 515 disks on such a system
(give or take a few). 11 - 515 disks is a fairly wide range
of storage possibilities.
As for making the comparison; benchmark. Find a way to
produce a comparible load on the system which you wish to
compare. Then find a way to measure the performance of
those components you find interesting. Run the test on
system of interest and compare the results.
re: Load average.
That is a pretty crude measurement for a database system.
How about "Response Time"? If the database is used interactively,
then the time for a user to get a response from the system is
very important. Raw transation rate doesn't typically include
a response time part.
Or "Transaction rate, weighted by cost". You can get a very
high transaction rate and good response time if you throw
enough money at the problem. But can you afford to? The
TPC benchmarks require that cost be figured into the audited
> results.
------------------------------------------------------------------------
-----------------
Dave Cherkus:
I hate to bring bad news, but I am of the opinion that it is
difficult to make meaningful comparisons between arbitrarily
configured machines, especially in the database arena.
In the case of the 3000/500 its attributes are:
- quite good main memory bandwidth (for its time)
- quite good cpu throughput (for its time)
- efficient but simplistic IO bus (requires a fair amount
of host cpu cycles to move data across the IO bus)
- good IO expandability (six turbochannel slots, two
builtin scsi, etc.)
- no CPU expandability (i.e. it's a uniprocessor)
Database software is very resource intense. It will take every
ounce of resource you throw at it and ask for more.
One meaningful comparison for database performance are the TPC
benchmarks, but one big problem is the benchmark run eats so many
resources (disks, IO controllers, memory, etc.) that it is very
expensive to run. This means that companies only run the benchmark for
very high end systems that will sell well in the database market.
Unfortunately the 3000/500 is a 'workstation', not a 'server' (in DEC's
eyes) so I doubt there is a TPC result for it.
------------------------------------------------------------------------
------------------
Robert L. McMillin:
No, it isn't crap, and yes, it is difficult. Different implementations
of the allegedly "portable" operating system Unix make it damn near
impossible to make reasonable comparisons. As with translations in
natural languages, the grace of the result is dependent on the skill
and knowledge of the translator.
> * How would one go about making such comparisons?
It's extremely difficult. Even well-defined things like SQL engines
require very, very rigorous test setups costing thousands of dollars and
months of effort (usually, they spend about a week on each vendor).
> * I was only after some crude indication such as "Load Average" with
> certain amount of active users - what other performance criteria would
> you look for?
Oh, heavens. There's a very good O'Reilly book on tuning Oracle
databases that covers this topic, but the thing you probably want to be
most aware of is query response time. The ideal, of course, is
instantaneous, but it turns out that there are a number of operations
best left as batch (i.e. one or more nighttime jobs). Like everything
else with this complicated subject, you don't know until you get there.
Again - Many thanks and have a GREAT Christmas & New Year
Received on Fri Dec 20 1996 - 00:41:20 NZDT