NFS caching problem?

From: Aaron J. Leliaert <"Aaron>
Date: Fri, 05 Feb 1999 15:29:20 -0800

We are having a problem with slow writes to an NFS mounted drive. For your
info, the NFS server is a Solaris 2.5 box.
The DEC's just don't write to the NFS mounted drive fast at all, and our
Sun machines write by a factor of 10 faster to the NFS mounted drive. Well,
here is a bettet desription of the problem by one of our developers.

----------------------

In a nutshell, we're writing large amounts of data to NFS via
fwrite(3). We're expecting the reading/writing of a fixed amount of
data to be much faster when the data is written as 1 or some small
number of large writes, as opposed to too many small writes.

On a Sun, reading and writing becomes exponentially faster as you move
data in larger blocks, both in terms of elapsed time and cpu usage.
Less system calls, faster. As expected. Here's output from a little
test program I wrote to examine this problem:

Test size 1500000 bytes, host harpoon (SunOS sun4u)
Test BlockSize Num Blocks Read (us) Write (us) ReadCpu WriteCpu
   2 17 88235 2418696 1960154 3699852 3183206
   3 71 21126 590571 556335 949962 749970
   4 295 5084 147855 234256 233324 216658
   5 1224 1225 39034 39059 49998 49998
   6 5077 295 12731 16260 33332 16666
   7 21050 71 9672 12700 0 16666
   8 87271 17 9434 12265 0 16666
   9 361811 4 10712 14975 16666 16666
  10 1500000 1 9007 10835 0 16666
aaron_at_harpoon[44]%

But on our dec, the read side gets faster, and cpu usage goes way
down for writing. But the writes are still very costly in terms of
elapsed time. fwrite() calls are taking their time:

Test size 1500000 bytes, host wheel.storm.com (OSF1 alpha)
Test BlockSize Num Blocks Read (us) Write (us) ReadCpu WriteCpu
   2 17 88235 4204101 4099610 3233204 1949922
   3 71 21126 769531 2522461 783302 483314
   4 295 5084 224609 2041992 183326 133328
   5 1224 1225 41992 3408203 33332 66664
   6 5077 295 21484 1909180 16666 49998
   7 21050 71 13672 1555664 16666 16666
   8 87271 17 13672 1495117 0 33332
   9 361811 4 21485 1874023 0 33332
  10 1500000 1 25390 1425782 33332 33332
aaron_at_wheel.storm.com[44]%

The write time is cut into a third to 1400ms, but we're hoping for 10ms
like the sun example. I'm assuming there is some kernel setting which
is preventing write buffering in Digital Unix... Unforunately I'm not a
DEC guru, and I can't seem to find any around the office right now.

Any advice would be greatly appreciated!


-grant
Received on Fri Feb 05 1999 - 23:33:54 NZDT

This archive was generated by hypermail 2.4.0 : Wed Nov 08 2023 - 11:53:39 NZDT