SUMMARY: File System Full

From: Gary George <GGeorge_at_edc.gov.ab.ca>
Date: Thu, 08 Jul 1999 13:30:33 -0600

>From the speed of response -- eight answers in fifteen minutes --
I conclude this is a sensitive area for all of us :-)
[I can't count the robot that answered immediately, "I'm on vacation;
if your problem is urgent, ...]

The suggested causes and remedies were were:
1) Any chance there were two links to the file and you only removed one?
   [I don't believe so, but it's something to put on the checklist.]

2) The usual cause of this is a file being held open by some process.
   Deleting the file merely removes the name, since it is still open.
   a) Was the offending file a log file? You need to make sure that
      a process still doesn't have hold of the file handle.
   b) Shutting down to single user would usually kill off all the
      processes, closing the file and freeing the space. Did df(1)
      report the high space usage in single-user? If so, you may
      have gotten the wrong file or the process was still running.
   c) If you use lsof on /usr, it will report the pid that is, hopefully,
      still holding on to that disk space. If you don't have a copy of lsof

      (list open files), you can obtain one at
      ftp://vic.cc.purdue.edu/pub/tools/unix/lsof/.


3) Problem with ADVFS [no,it's UFS]- turn on quotas by changing fstab, eg:
        usr_old#usr /mnt2 advfs rw,userquota,groupquota 0 0

   Then unmount and remount the file system, and run quotacheck:
        quotacheck -v
   Look at the filesystem and it should have gained back all of it's
   missing space. [This I'll try, just for my own education.]

4) You might want to check the mount point itself to see if there is any
   "garbage" underneath the mounted filesystem.
   [This I'll also look for.]

5) See if you have a trashcan defined, if you do, instead of deleting files
   completely, they are moved to the trashcan directory.

6)Use
        find /usr -size +5000 -exec ls -l {} \;
  to find if something is going crazy creating big files and space gets
  consumed as soon as it's free...

(2a) hit the nail on the head, and (6) is what found it for me ... we'd just

put in C2 security, and somehow (nobody's admitting to doing it
deliberately)
auditing got turned on. Auditing of *everything*.


Thanks to:
        Burch Seymour
        Stephen Spalding
        Roy Smith
        alan_at_nabeth.cxo.dec.com
        Walter Alpuin
        Brian Schrann
        Bill Boddy
        Roger Leonard
        Lisa A. Wallace
        Patty Jahnz
        Bryan Lavelle
        Alvarez, Cynthia
        Brian Hostetter
        Hank Lee

Gary George, I.S.P.
Systems Administrator
Information Services, Alberta Learning
780-427-3512

Original message:
I have a problem: /usr, on a UFS partition, filled up to 102%.

I found and erased the offending file, but df still reported 102% full.

I took it down to single user mode and fsck'd it (-o -p), but after mounting
it still reports full.

What's next?
Received on Thu Jul 08 1999 - 19:32:48 NZST

This archive was generated by hypermail 2.4.0 : Wed Nov 08 2023 - 11:53:39 NZDT