Yesterday my server began to have various issues (500 errors, couldn’t login to sites, etc). After some looking, I found out that, despite the fact there was still 13% capacity on the hard drive itself, all of my inodes were used up. My server is set to a maximum of 1 million. Not having experienced this before, I was at a loss.
root@host [/var/spool/exim]# df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/vzfs 1000000 1000000 0 100% / none 98304 96 98208 1% /dev
After Googling the issue, I learned inodes are actually individual files. So I started going through and removing sites, emails, databases, etc of files that were no longer needed on the site. I did this for quite some time, barely freeing up 50,000 inodes. This filled up again very quickly.
I then decided to search for directories that had a lot of files. I found the following on stackoverflow.com:
find . | cut -d/ -f 2 | uniq -c
I started from / as the root user, and slowly worked my way into /var/spool/exim/. In there, in the msglog/ and input/ folder were about 600,000 inodes. So I simply emptied those directories:
rm -rf /var/spool/exim/msglog/* rm -rf /var/spool/exim/input/*
Once I did that, I was set:
root@host [/var/spool/exim]# df -i Filesystem Inodes IUsed IFree IUse% Mounted on /dev/vzfs 1000000 363453 636547 37% / none 98304 96 98208 1% /dev
I also found this, so beware:
The danger of doing [what is listed above] is that you will also delete any email in transit that should be delivered. So, you ought to stop exim first, then use the WHM option to delete the queue and then restart exim. Then, of course, you need to fix the problem that’s causing the huge queue, starting with checking that accounts are not using :blackhole: and are using :fail: instead.
A safer way to do this may be here: