Greetings, Managers--
Several weeks ago, I posted a question about impossible free space
statistics reported by the AdvFS defragment utility on a DU 3.0 system;
I got only two responses, which while reassuring, didn't contain the
definitive answer I had been hoping for. So I decided to hold off on
a summary, while I kept an eye on the defragmentation statistics for a
bit longer.
Having done that, however, I'm none the wiser, so I can really only say
that nothing has broken yet, and refer the interested reader to Tom Blinn's
comments for a plausible explanation.
Thanks to Sean O'Connell and Dr. Thomas P. Blinn; the short version of
my question is repeated below (check the archive for the long version,
if you really have to), followed by their suggestions:
> Today I found some rather suspect free space statistics in the output
> from a defragment run on an AdvFS domain:
>
> Oxter> more data_defrag.log
> defragment: Defragmenting domain 'data_domain'
>
> Pass 1; Clearing
> Volume 2: area at block 7771040 ( 2610144 blocks): 41% full
> Domain data as of the start of this pass:
> Extents: 11912
> Files w/extents: 11831
> Avg exts per file w/exts: 1.01
> Aggregate I/O perf: 100%
> Free space fragments: 556
> <100K <1M <10M >10M
> Free space: -67% 6% 38% 123% <====
> Fragments: 308 146 72 30
>
> Filling
> Current domain data:
> Extents: 11832
> Files w/extents: 11831
> Avg exts per file w/exts: 1.00
> Aggregate I/O perf: 100%
> Free space fragments: 317
> <100K <1M <10M >10M
> Free space: -68% 4% 23% 141% <====
> Fragments: 168 94 39 16
>
> defragment: Defragmented domain 'data_domain'
>
> So far we have seen no problems with data corruption (only the odd
> free space statistics), but I'm concerned that this might be the first
> signs of a a problem. Any comments or pointers to documentation would
> be welcome.
----- From Sean O'Connell:
> Hi, Bill
>
> Just a thought, did you set your hard limit on you fmdn/fsets. IF
> it is set based on 95% of the smaller drive, you *will* get bogus
> results (or it makes sense that you might).
>
> What is the output of showfsets? You can use the chfsets -b command
> to set that variable. It might just be confused because it still
> views the upper limit of the disk as being based on the 4.3GB drive.
> To prevent problems defragmenting, you want 5% free, so I set my
> hard limit to being: 0.95*(size in 512k blocks)/2. For some reason
> I think chfsets wants 1024k block data...good ol dec.
Sorry, I forgot to mention that had corrected the fileset quotas after
adding the new disk to the domain; by the time I ran the first defragment
job, the sum of the fileset hard quotas was essentially the same as the
size of the domain.
> Not sure if this is the case or not...
>
> S
> --
> -------------------------------------------------------------------------
> Sean O'Connell Email: sto_at_stat.Duke.EDU
> Institute of Statistics and Decision Sciences Phone: (919) 684-5419
> Duke University Fax: (919) 684-8594
----- From Dr. Thomas P. Blinn
> Bill,
>
> I've left your comprehensive description at the end of this message
[deleted from the summary]
> since I'm blind-copying our file systems product manager as well as our
> file systems engineering team leader. You are certainly running an old
> version of Digital UNIX and AdvFS. LOTS of work has been done on all
> of the AdvFS utilities, as well as on what's in the base system, since
> the V3.0 release came out.
Yes, that's why I thought I'd try a post to the list ... I've meanwhile
bumped the upgrade up a few places on my priority list ...
> As you say, the "Free space" numbers being reported are clearly wrong.
> The only explanation I can offer is that there must be some code in the
> defrag utility that was, perhaps, incorrectly interpreting a 64 bit
> value as a 32 bit one, and with certain data, the "sign bit" is set and
> the number looks negative when it's really positive. You can do this
> in the "C" language much more easily than should be the case. Clearly,
> there is no way that a negative percentage of the free space fragments
> is in chunks less than 100K in size.
>
> I suspect that as you run defragment, eventually the statistics will
> straighten themselves out again. In any case, the fact that msfsck and
> vchkdir aren't complaining is a good sign. If things really were bad,
> you'd see complaints. And migrating everything from the old disk to
> the new one should have defragmented the file domain anyway.
Thanks -- those comments about msfsck and vchkdir were reassuring, so for
a week or two after my original post I continued to make daily defragment
runs on this domain (mainly to see the statistics, not because they were
needed -- it is correct that the domain was largely defragmented at this
point).
But FWIW, there was no simple trend to the free space statistics to
be seen: for the first several days, the statistics got worse (larger
negative percentages for the <100K bin, larger positive percentages for
the >10M bin), until one day (for no reason that I could see) the "Domain
data as of the start of this pass:" were normal; but then (starting with
the "Current domain data:" after that same pass) I started to get worse
again.
After tiring of that game, I dropped the frequency of the defragment runs
to once a week, and since then I have consistently seen normal Free space
statistics (i.e., all zero or positive percentages adding to 100%) in the
"Domain data as of the start of this pass:" and bogus statistics in the
"Current domain data:" at the end of the run, with the degree of "bogosity"
varying wildly from run to run for no obvious reason. Just for grins, here
is the worst case seen so far:
Free space fragments: 406
<100K <1M <10M >10M
Free space: -24776% 1050% 4497% 19329%
Fragments: 93 193 82 38
The distribution of Fragments has always added up correctly (as have the
Free space percentages, for that matter) and the defragmentation per se
appears to work, so you are probably right that it is just a cosmetic
problem with the reporting of Free space percentages.
> I'd have to go back and confirm that under V3.0 of Digital UNIX and of
> the AdvFS software, a 9GB disk drive was supported. I'm not certain it
> was; if not, it may well work OK, but that could throw off some of the
> statistics.
>
> The only other thing I can suggest is to check whether there are any
> patches available for those old, outmoded utilities, but I don't think
None that I could find...
> we still support that version; you really should be thinking about an
> update to a newer version (at least to the V3.2G release, and we're up
> to V4.0C and about to release V4.0D in the winter).
>
> Tom
>
> Dr. Thomas P. Blinn, UNIX Software Group, Digital Equipment Corporation
> 110 Spit Brook Road, MS ZKO3-2/U20 Nashua, New Hampshire 03062-2698
> Technology Partnership Engineering Phone: (603) 881-0646
> Internet: tpb_at_zk3.dec.com Digital's Easynet: alpha::tpb
> ACM Member: tpblinn_at_acm.org PC_at_Home: tom_at_felines.mv.net
>
> Worry kills more people than work because more people worry than work.
>
> Keep your stick on the ice. -- Steve Smith ("Red Green")
>
> My favorite palindrome is: Satan, oscillate my metallic sonatas.
> -- Phil Agre, pagre_at_ucsd.edu
>
> Opinions expressed herein are my own, and do not necessarily represent
> those of my employer or anyone else, living or dead, real or imagined.
Received on Fri Oct 10 1997 - 13:56:20 NZDT