Back around the 19th of July I posed the following to the group:
=============================================================================
>
> I'm seeing a performance problem I don't understand. I'm doing benchmarks to
> compare a DEC alphastation 600 5/266 to an SGI R8000 Power Indigo 2.
> It's a wonderful box by the way! The DEC is looking very good,
> but in some fairly repeatable circumstances, it goes to pot. Vmstat will
> show it running along at 99% cpu in user mode and suddenly that drops down to
> 1-5 %, with over 90% idle. The cpu looks like it is in never never land. At
> this point, the mouse and keyboard are frozen, and nothing I do there works,
> although I have a vt220 with the vmstat running, and it updates normally.
> I was shading a proE model once when this happened, and the partially
> shaded model (an 88MB model, quite large, with about 500 MB out of 1.2GB
> of swap used) just hung on the screen for about 2 minutes, then it
> snapped back to life and resumed as though nothing had happened.
> The system is loaded with 256MB of memory, although it is swapping
> extensively, and when the "freeze" occurs, there are only about 20
> pages of free memory. Anyone got any ideas what is happenning?
> It's killing the machine in the benchmarks.
>
> Stan Huhman
> John Deere PEC
> Waterloo, Iowa
==============================================================================
The responses I received were quite useful, and put me on to several ideas:
From: Benoit Maillard - Digital France
It can be your case:
Default Digital Unix behaviour is to allow the Unified Buffer Cache
to grow up to 100% of available memory. So intense IO traffic pushes
all program to swap. In this case, the machine seems to hang.
"Correct" (more balanced) behaviour is to limit the space available
to the UBC.
sysconfig -q vm will tell you the current ubcmaxpercent value.
Put a lower value in /etc/sysconfigtab if you want to modify it.
The full explanation is in the "System tuning and Perf.guide"
Regards,
Benoit Maillard
maillard_at_fgt.dec.com
From: Tom Barkanic <intrepid!dino!tb_at_uunet.uu.net>
What kind of swap mode are you using? Do you have a /sbin/swapdefault file? If
you do I have heard of people having problems with this type of swap mode. On
my systems I renamed /sbin/swapdefault and rebooted to use the 'lazy' type of
swap mode.
From: alan_at_nabeth.cxo.dec.com
Sounds like the large process is getting swapped out at
some point. Assuming that a single swap disk can support
3000 KB/sec, you expect a 200 MB address space to take
60 or seconds to move one direction (to swap) and another
60 or seconds to move in the other (to memory).
I'd use something to watch the disk I/O to the page/swap
device(s) during the freeze. If the disk(s) is/are busy,
that would suggest that swapping it in/out may be the
problem. I don't know how to re-tune the system to
convince not to swap out such a large process, but there
is a tuning document in our doc set.
From: Allan Small (Cafeine propelled) <small_at_gidday.ENET.dec.com>
Hi Stan,
It looks like a severe memory shortage problem. Posb. the process is being
swapped out.
What are you using to monitor the system? I trust you are using defered swaping
(no /sbin/swapdefault). I would increase the page free target for the page
stealing daemon:
eg:
# dbx -k /vmunix /dev/mem
(dbx) assign page_free_target=400 (in the running kernel)
or
Add the following stanza to the /etc/sysconfigtab file and reboot
vm:
vm-page-free-target = 400
The default value for this parameter is 128 pages. For applications that are
memory intensive, the Alpha is capable of consuming those 128 pages quickly, and
the page stealing daemon is frequently started. I generally increase the page
free target until that the page stealing daemon is run less frequently (and the
free list is maintained at close to the target value.
FWIW, if the size of the model + application is larger than your free memory,
you may end up testing the performance of your IO subsystem, rather than the
CPU.
All the best
Allan
=============================================================================
Conclusion:
It seems apparent that the machine is going in to a swapping mode.
As Mr. Small suggests, when that happens we are testing the performance of
the I/O system, but hey, that's part of performance. I had been
using immediate mode swapping, because in past experience, deferred (or lazy)
mode killed processes when you really do run out of swap space. We use a
lot of swap space, ProEngineer can take over 1GB per session. After
switching to deferred mode, we did note a performance improvement and I
will be switching all my machines to use this, after ensuring that I have
adequate swap space setup.
Also, it appears that advfs was causing a performance problem, not because
writing to an advfs fileset is slow, but because the 18MB of memory it
consumes, in one of our benchmarks, sent us over the 256MB of system
memory into swapland.
The two kernel parameters that were mentioned as having a potential impact
were also adjusted, and seemed to produce positive results. It's somewhat
agravating to see benchmark results vary when the exact same problem is run,
this seemed to happen a lot when we saw swapping occur, I assume because
of the variability of the I/O subsystem.
We also started our benchmarks with an RZ28 system disk , with 425MB swap
space, 500MB swap on an rz26, and the remainder of that rz26, another full
one, and an RZ29 combined into the advfs fileset. Results improved
significantly when we removed the 2 RZ26's and 1GB of a seagate ST15150N
for secondary swap, and combined the RZ29 with another seagate for the advfs
set.
Hopefully, this puts running benchmarks behind me for a good long while.
It seems thre is always at least one more case that needs to be run to
understand things. By the way, for our application, the alphastation 600
5/250 outperforms the SGI power Indigo II Extreme. Suprisingly, perhaps,
the ZLXp-L2 graphics performs close to the Extreme. Perhaps even more
surprising, the ZLXp-E3 graphics is for many graphic operations at pretty
much the same performance level too. On the other hand, SGI touted their
XZ graphics as good, but it fell off the performance curve fairly
dramatically. With the alphastation 600 5/300 already announced with
pricing set, it seems DEC has a winning combination of high end
workstations. Oh, the 5/250 pricing is less than the SGI Extreme too.
Better performance, lower price, a pretty hard combination to beat!
Received on Wed Aug 02 1995 - 00:10:44 NZST