Document Actions

File System Full

Detection and correction of full filesystems

Detection of full GFS filesystem

GFS filesystem format

GFS filesystem block are seperated into metadata and data blocks. Data blocks are converted into metadata blocks when needed. A metadata block that is not needed anymore is not automatically converted into data block. It stays in the free metadata block section.

df is wrong

The df tool returns an overall value for GFS data and metadata blocks. This can lead to wrong information.

df Example :

         [root@stage1 ~]# df -h /cluster/pool/media_stage/
         Filesystem            Size  Used Avail Use% Mounted on
         /dev/VG_MEDIA_STAGE/LV_MEDIA_STAGE
                      9.4G  7.1G  2.3G  76% /cluster/pool/media_stage

gfs_tool is right

To get the correct information the command gfs_tool df should be used.

gfs_tool df Example:

         STAGE-CLUSTER [root@stage1 ~]# gfs_tool df /cluster/pool/media_stage/
         /cluster/pool/media_stage:

         [...]

           Type           Total          Used           Free           use%
           ------------------------------------------------------------------------
           inodes         75023          75023          0              100%
           metadata       602625         6417           596208         1%
           data           1760304        1758484        1820           100%

Correcting a Full File System Condition

Converting Metadata blocks to Data blocks

To use free metadata blocks, they have to be converted to data blocks. This can be done with the command *gfs_tool reclaim *

Example:

         [root@stage1 ~]# gfs_tool reclaim /cluster/pool/media_stage/
         Don't do this if this file system is being exported by NFS (on any machine).

         Are you sure you want to proceed? [y/n] y

         Reclaimed:
         version 0
         inodes 0
         metadata 596178

After this, all free metadata block should be converted to free data blocks.