Usually, to delete/remove a file from a Linux terminal, we use the rm command (delete files), shred command (securely delete a file), wipe command (securely erase a file), or secure-deletion toolkit (a collection of secure file deletion tools).
We can use any of the above utilities to deal with relatively small files. What if we want to delete/remove a huge file/directory say about 100-200GB?
This may not be as easy as it seems, in terms of the time taken to remove the file (I/O scheduling) as well as the amount of RAM consumed while carrying out the operation.
In this tutorial, we will explain how to efficiently and reliably delete huge files/directories in Linux.
Suggested Read: 5 Ways to Empty or Delete a Large File Content in Linux
The main aim here is to use a technique that will not slow down the system while removing a huge file, resulting to reasonable I/O. We can achieve this using the ionice command.
Deleting HUGE (200GB) Files in Linux Using ionice Command
ionice is a useful program that sets or gets the I/O scheduling class and priority for another program. If no arguments or just
-p is given, ionice will query the current I/O scheduling class and priority for that process.
If we give a command name such as rm command, it will run this command with the given arguments. To specify the process IDs of running processes for which to get or set the scheduling parameters, run this:
# ionice -p PID
To specify the name or number of the scheduling class to use (0 for none, 1 for real-time, 2 for best-effort, 3 for idle) the command below.
This means that rm will belong to the idle I/O class and only uses I/O when any other process does not need it:
---- Deleting Huge Files in Linux ----- # ionice -c 3 rm /var/logs/syslog # ionice -c 3 rm -rf /var/log/apache
If there won’t be much idle time on the system, then we may want to use the best-effort scheduling class and set a low priority like this:
# ionice -c 2 -n 6 rm /var/logs/syslog # ionice -c 2 -n 6 rm -rf /var/log/apache
Note: To delete huge files using a secure method, we may use the shred, wipe and various tools in the secure-deletion toolkit mentioned earlier on, instead of the rm command.
Suggested Read: 3 Ways to Permanently and Securely Delete Files/Directories’ in Linux
For more info, look through the ionice man page:
# man ionice
That’s it for now! What other methods do you have in mind for the above purpose? Use the comment section below to share with us.
8 thoughts on “How to Delete HUGE (100-200GB) Files in Linux”
This only work do you are using cfq as io scheduler
We’ll check this out. Thanks for the feedback.
Why not simply use “nice” and set a low priority? Would that be sufficiently efficient? Linux will know how to handle the priorities so when the cpu is not running high, it will give more cpu time to the rm command. I don’t see any significant difference.
A reasonable way to accomplish the job, we’ll surely test this out. Many thanks for writing back.
echo ” ” > huge_file && rm huge_file
Okay, don’t you think without using ionice, this command will cause high I/O loads and gradually slow down the system? Thanks for sharing this.
No – command like:
No take many I/O
This is short way to make zero size file in bash
Okay, thanks for the useful tip.