Refresh Linux disk space

Sometimes even if you deleted the large files for example log files created by Weblogic server, Apache Tomcat, Mysql and others disk space is not available in the system. It is because the file is being used by these applications and not able to release the space occupied by file.

In that case you have to identify the processes using these deleted files. To do so you can take the help of linux proc directory which is pseudo-file system used as an interface to kernel data structure. proc directory contains /proc/[pid]/fd subdirectory.

This subdirectory containing one entry for each file which  the  process  has open,  named  by  its  file descriptor, and which is a symbolic link to the actual file.  Thus, 0 is standard input, 1 standard output, 2 standard error etc.

Alright lets begin the work.

1. Type the following command in your bash shell

 ls -la /proc/*/fd/* 2>&1 | grep deleted  

Where

• 2 is the default file descriptor for stderr.
• 1 is the default file descriptor for stdout.
• >& is shell syntax for "fold a file descriptor into another"

Here 2>&1 means redirect stderr(2) to stdout(1). At first, 2>1 may look like a good way to redirect stderr to stdout.  However, it will actually be interpreted as "redirect stderr to a file named 1". & indicates that what follows is a file descriptor and not a filename. So the construct becomes: 2>&

Output:

lrwx------ 1 root   root     64 Mar  7 03:20 /proc/1437/fd/3 -> /tmp/vmware-root/appLoader-1437.log (deleted)

lrwx------ 1 oracle oinstall 64 Mar  7 03:20 /proc/7550/fd/3 -> /tmp/vmware-oracle/appLoader-7550.log (deleted)

l-wx------ 1 root root 64 Mar 26 04:27 /proc/3675/fd/340 -> /usr/local/WebLogic/wlApps/mi/mi.8202/WebContent/WEB-INF/log/spring.log (deleted)

2. Here we identify 3 process ids 1437 ( from /proc/1437/fd/3), 7550 (from /proc/7550/fd/3) and 3675 (/proc/3675/fd/340) . We can grep these processes to identify which program actually using these files. Let's grep one process.

ps aux | grep 3675 

Output:

root      3675  0.5 14.8 5573700 2448684 ?     Sl   Mar03 200:54 /usr/local/WebLogic/10.3.0.0/jrockit-R27.5.0-jdk1.6.0_03/bin/java -jrockit -Xms2g -Xmx2g -Xverify:none -da -Dplatform.home=/usr/local/WebLogic/10.3.0.0/wlserver_10.3 -Dwls.home=/usr/local/WebLogic/10.3.0.0/wlserver_10.3/server -Dweblogic.home=/usr/local/WebLogic/10.3.0.0/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/usr/local/WebLogic/10.3.0.0/patch_wlw1030/profiles/default/sysext_manifest_classpath:/usr/local/WebLogic/10.3.0.0/patch_wls1030/profiles/default/sysext_manifest_classpath:/usr/local/WebLogic/10.3.0.0/patch_cie660/profiles/default/sysext_manifest_classpath -Dweblogic.Name=mi.8202 -Djava.security.policy=/usr/local/WebLogic/10.3.0.0/wlserver_10.3/server/lib/weblogic.policy weblogic.Server

Output shows Weblogic Server version 10.3.0.0 using the file /usr/local/WebLogic/wlApps/mi/mi.8202/WebContent/WEB-INF/log/spring.log.

3. Now we have to kill this process to regain our space.

 kill -9 3675