LFCS #3: How to Archive Files, Set File Permissions and Finding Files in Linux

Recently, the Linux Foundation started the LFCS (Linux Foundation Certified Sysadmin) certification, a brand new program whose purpose is to allow individuals to perform basic to intermediate system administration tasks on Linux systems.

This includes supporting already running systems and services, along with first-level troubleshooting and analysis, plus the ability to decide when to escalate issues to engineering teams.

The series will be titled Preparation for the LFCS (Linux Foundation Certified Sysadmin) Parts 1 through 33 and cover the following topics:

No. Title
Part 1 How to Use ‘Sed’ Command to Manipulate Files in Linux
Part 2 How to Install and Use Vi/Vim in Linux
Part 3 How to Archive Files, Set File Permissions, and Finding Files in Linux
Part 4 Partitioning Storage Devices, Formatting Filesystems, and Configuring Swap Partition
Part 5 Mount/Unmount Local and Network (Samba & NFS) Filesystems in Linux
Part 6 Assembling Partitions as RAID Devices – Creating & Managing System Backups
Part 7 Managing System Startup Processes and Services (SysVinit, Systemd, and Upstart
Part 8 How to Manage User & Groups, File Permissions, and Sudo Access
Part 9 Linux Package Management with Yum, RPM, Apt, Dpkg, Aptitude, and Zypper
Part 10 Learning Basic Shell Scripting and Filesystem Troubleshooting
Part 11 How to Manage and Create LVM Using vgcreate, lvcreate, and lvextend Commands
Part 12 How to Explore Linux with Installed Help Documentations and Tools
Part 13 How to Configure and Troubleshoot Grand Unified Bootloader (GRUB)
Part 14 Monitor Linux Processes Resource Usage and Set Process Limits on a Per-User Basis
Part 15 How to Set or Modify Kernel Runtime Parameters in Linux Systems
Part 16 Implementing Mandatory Access Control with SELinux or AppArmor in Linux
Part 17 How to Set Access Control Lists (ACLs) and Disk Quotas for Users and Groups
Part 18 Installing Network Services and Configuring Automatic Startup at Boot
Part 19 An Ultimate Guide to Setting Up FTP Server to Allow Anonymous Logins
Part 20 Setup a Basic Recursive Caching DNS Server and Configure Zones for Domain
Part 21 How to Install, Secure, and Performance Tuning of MariaDB Database Server
Part 22 How to Install and Configure NFS Server for File System Sharing
Part 23 How to Setup Apache with Name-Based Virtual Hosting with SSL Certificate
Part 24 How To Setup an Iptables Firewall to Enable Remote Access to Services in Linux
Part 25 How to Turn a Linux into a Router to Handle Traffic Statically and Dynamically
Part 26 How to Setup Encrypted Filesystems and Swap Using Cryptsetup Tool
Part 27 How to Monitor System Usage, Outages, and Troubleshoot Linux Servers
Part 28 How to Setup a Network Repository to Install or Update Packages
Part 29 How to Audit Network Performance, Security, and Troubleshooting
Part 30 How to Install and Manage Virtual Machines and Containers
Part 31 Learn the Basics of Git to Manage Projects Efficiently
Part 32 A Beginner’s Guide to Configuring IPv4 and IPv6 Addresses in Linux
Part 33 A Beginner’s Guide to Creating Network Bonding and Bridging in Ubuntu

This post is Part 3 of a 33-tutorial series, here in this part, we will cover how to archive/compress files and directories, set file attributes, and find files on the filesystem, that are required for the LFCS certification exam.

Archiving and Compression Tools for Linux

A file archiving tool groups a set of files into a single standalone file that we can back up to several types of media, transfer across a network, or send via email.

The most frequently used archiving utility in Linux is the tar command. When an archiving utility is used along with a compression tool, it allows to reduce the disk size that is needed to store the same files and information.

Linux tar Utility

tar bundles a group of files together into a single archive (commonly called a tar file or tarball). The name originally stood for tape archiver, but we must note that we can use this tool to archive data to any kind of writeable media (not only to tapes).

Tar is normally used with a compression tool such as gzip, bzip2, or xz to produce a compressed tarball.

The basic syntax of the tar command is as follows:

# tar [options] [pathname ...]

Where ... represents the expression used to specify which files should be acted upon.

Most Commonly Used Tar Commands

Long option Abbreviation Description
 –create  c  Creates a tar archive
 –concatenate  A  Appends tar files to an archive
 –append  r  Appends files to the end of an archive
 –update  u  Appends files newer than copy in archive
 –diff or –compare  d  Find differences between archive and file system
 –file archive  f  Use archive file or device ARCHIVE
 –list  t  Lists the contents of a tarball
 –extract or –get  x  Extracts files from an archive

Normally Used tar Operation Modifiers

Long option Abbreviation Description
 –directory dir  C  Changes to directory dir before performing operations
 –same-permissions  p  Preserves original permissions
 –verbose  v  Lists all files read or extracted. When this flag is used along with –list, the file sizes, ownership, and time stamps are displayed.
 –verify  W  Verifies the archive after writing it
 –exclude file  —  Excludes files from the archive
 –exclude=pattern  X  Exclude files, given as a PATTERN
 –gzip or –gunzip  z  Processes an archive through Gzip
 –bzip2  j  Processes an archive through bzip2
 –xz  J  Processes an archive through xz

Linux Gzip, Bzip2, and Xz Utilities

Gzip is the oldest compression tool and provides the least compression, while bzip2 provides improved compression. In addition, xz is the newest but (usually) provides the best compression.

The advantages of best compression come at a price: the time it takes to complete the operation, and the system resources used during the process.

Normally, tar files compressed with these utilities have .gz, .bz2, or .xz extensions, respectively. In the following examples, we will be using these files: file1, file2, file3, file4, and file5.

Compressing Files with gzip, bzip2 and xz

Group all the files in the current working directory and compress the resulting bundle with gzip, bzip2, and xz (please note the use of a regular expression to specify which files should be included in the bundle – this is to prevent the archiving tool to group the tarballs created in previous steps).

# tar czf myfiles.tar.gz file[0-9]
# tar cjf myfiles.tar.bz2 file[0-9]
# tar cJf myfile.tar.xz file[0-9]
Compress Multiple Files Using tar
Compress Multiple Files

List Contents and Updating/Appending Files Tar Archive

List the contents of a tarball and display the same information as a long directory listing. Note that update or append operations cannot be applied to compressed files directly.

Decompress Tar Files

If you need to update or append a file to a compressed tarball, you need to uncompress the tar file and update/append to it, then compress it again.

# tar tvf [tarball]
Check Files in tar Archive
List Archive Content

Run any of the following commands:

# gzip -d myfiles.tar.gz	[#1] 
# bzip2 -d myfiles.tar.bz2	[#2] 
# xz -d myfiles.tar.xz 		[#3] 

Delete or Add Files to Tar Archive

# tar --delete --file myfiles.tar file4 (deletes the file inside the tarball)
# tar --update --file myfiles.tar file4 (adds the updated file)

and

# gzip myfiles.tar		[ if you choose #1 above ]
# bzip2 myfiles.tar		[ if you choose #2 above ]
# xz myfiles.tar 		[ if you choose #3 above ]

Finally,

# tar tvf [tarball] #again

and compare the modification date and time of file4 with the same information as shown earlier.

Exclude Files from Backups

Suppose you want to perform a backup of the user’s home directories. A good sysadmin practice would be (may also be specified by company policies) to exclude all video and audio files from backups.

Maybe your first approach would be to exclude from the backup all files with an .mp3 or .mp4 extension (or other extensions). What if you have a clever user who can change the extension to .txt or .bkp, your approach won’t do you much good.

In order to detect an audio or video file, you need to check its file type with a file. The following shell script will do the job.

#!/bin/bash
# Pass the directory to backup as first argument.
DIR=$1
# Create the tarball and compress it. Exclude files with the MPEG string in its file type.
# -If the file type contains the string mpeg, $? (the exit status of the most recently executed command) expands to 0, and the filename is redirected to the exclude option. Otherwise, it expands to 1.
# -If $? equals 0, add the file to the list of files to be backed up.
tar X <(for i in $DIR/*; do file $i | grep -i mpeg; if [ $? -eq 0 ]; then echo $i; fi;done) -cjf backupfile.tar.bz2 $DIR/*
Exclude Files in tar Archive
Exclude Files in the tar

Restore Backup with Tar Preserving Permissions

You can then restore the backup to the original user’s home directory (user_restore in this example), preserving permissions, with the following command.

# tar xjf backupfile.tar.bz2 --directory user_restore --same-permissions
Restore Files from tar Archive
Restore Files from the Archive

Using Find Command to Search for Files

The find command is used to search recursively through directory trees for files or directories that match certain characteristics, and can then either print the matching files or directories or perform other operations on the matches.

Normally, we will search by name, owner, group, type, permissions, date, and size.

The basic syntax of the find command is as follows:

# find [directory_to_search] [expression]

Finding Files Recursively According to Size

Find all files (-f) in the current directory (.) and 2 subdirectories below (-maxdepth 3 includes the current working directory and 2 levels down) whose size (-size) is greater than 2 MB.

# find . -maxdepth 3 -type f -size +2M
Find Files by Size in Linux
Find Files Based on Size

Finding and Deleting Files That Match Certain Criteria

Files with 777 permissions are sometimes considered an open door to external attackers. Either way, it is not safe to let anyone do anything with files. We will take a rather aggressive approach and delete them! (‘{}+ is used to “collect” the results of the search).

# find /home/user -perm 777 -exec rm '{}' +
Find all 777 Permission Files
Find Files with 777Permission

Finding Files Based on Timestamps

Search for configuration files in /etc that have been accessed (-atime) or modified (-mtime) more (+180) or less (-180) than 6 months ago or exactly 6 months ago (180).

Modify the following command as per the example below:

# find /etc -iname "*.conf" -mtime -180 -print
Find Files by Modification Time
Find Modified Files

Set File Permissions and Basic Attributes

The first 10 characters in the output of ls -l are the file attributes. The first of these characters is used to indicate the file type:

  • - : a regular file
  • -d : a directory
  • -l : a symbolic link
  • -c : a character device (which treats data as a stream of bytes, i.e. a terminal)
  • -b : a block device (which handles data in blocks, i.e. storage devices)

The next nine characters of the file attributes are called the file mode and represent the read (r), write (w), and execute (x) permissions of the file’s owner, the file’s group owner, and the rest of the users (commonly referred to as “the world”).

Whereas the read permission on a file allows the same to be opened and read, the same permission on a directory allows its contents to be listed if the execute permission is also set. In addition, the execute permission in a file allows it to be handled as a program and run, while in a directory it allows the same to be cd’ed into it.

File permissions are changed with the chmod command, whose basic syntax is as follows:

# chmod [new_mode] file

Where new_mode is either an octal number or an expression that specifies the new permissions.

The octal number can be converted from its binary equivalent, which is calculated from the desired file permissions for the owner, the group, and the world, as follows:

The presence of certain permission equals a power of 2 (r=22, w=21, x=20), while its absence equates to 0. For example:

Linux File Permissions
File Permissions

To set the file’s permissions as above in octal form, type:

# chmod 744 myfile

You can also set a file’s mode using an expression that indicates the owner’s rights with the letter u, the group owner’s rights with the letter g, and the rest with o.

All of these “individuals” can be represented at the same time with the letter a. Permissions are granted (or revoked) with the + or - signs, respectively.

Remove Execute Permission On a Script to All Users

As we explained earlier, we can revoke a certain permission by prepending it with the minus sign and indicating whether it needs to be revoked for the owner, the group owner, or all users. The one-liner below can be interpreted as follows: Change mode for all (a) users, revoke (-) execute permission (x).

# chmod a-x backup.sh

Granting read, write, and execute permissions for a file to the owner and group owner, and read permissions for the world.

When we use a 3-digit octal number to set permissions for a file, the first digit indicates the permissions for the owner, the second digit for the group owner and the third digit for everyone else:

  • Owner: (r=22 + w=21 + x=20 = 7)
  • Group owner: (r=22 + w=21 + x=20 = 7)
  • World: (r=22 + w=0 + x=0 = 4),
# chmod 774 myfile

In time, and with practice, you will be able to decide which method to change a file mode works best for you in each case. A long directory listing also shows the file’s owner and its group owner (which serve as a rudimentary yet effective access control to files in a system):

Linux File Listing
Linux File Listing

File ownership is changed with the chown command. The owner and the group owner can be changed at the same time or separately. Its basic syntax is as follows:

# chown user:group file

Where at least a user or group needs to be present.

Chown Command Examples

Changing the owner of a file to a certain user.

# chown gacanepa sent

Changing the owner and group of a file to a specific user:group pair.

# chown gacanepa:gacanepa TestFile

Changing only the group owner of a file to a certain group. Note the colon before the group’s name.

# chown :gacanepa email_body.txt
Conclusion

As a sysadmin, you need to know how to create and restore backups, how to find files in your system and change their attributes, along with a few tricks that can make your life easier and will prevent you from running into future issues.

I hope that the tips provided in the present article will help you to achieve that goal. Feel free to add your own information and ideas in the comments section for the benefit of the community. Thanks in advance!

The LFCS eBook is available now for purchase. Order your copy today and start your journey to becoming a certified Linux system administrator!

Product Name Price Buy
The Linux Foundation’s LFCS Certification Preparation Guide $19.99 [Buy Now]

Last, but not least, please consider buying your exam voucher using the following links to earn us a small commission, which will help us keep this book updated.

Hey TecMint readers,

Exciting news! Every month, our top blog commenters will have the chance to win fantastic rewards, like free Linux eBooks such as RHCE, RHCSA, LFCS, Learn Linux, and Awk, each worth $20!

Learn more about the contest and stand a chance to win by sharing your thoughts below!

Gabriel Cánepa
Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work.

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Join the TecMint Weekly Newsletter (More Than 156,129 Linux Enthusiasts Have Subscribed)
Was this article helpful? Please add a comment or buy me a coffee to show your appreciation.

12 Comments

Leave a Reply
    • @Nthapa,

      Thank you for your interest! You’re currently reading Part 3 of our series. If you’re looking for previous parts, you can find them in the table above.

      Reply
  1. Hi Gabriel,

    Thanks for the nice write-ups. I am new to Linux and scripting, could you help explain this your line of code for a newbie:

    tar X <(for i in $DIR/*; do file $i | grep -i mpeg; if [ $? -eq 0 ]; then echo $i; fi;done) -cjf backupfile.tar.bz2 $DIR/*
    Or could you re-write it in an expanded form?

    Thanks in anticipation.
    Chiadi

    Reply
    • @Chiadi,

      Gabriel already provided explanation of that command, see:

      "#!/bin/bash
      # Pass the directory to backup as first argument.
      DIR=$1
      # Create the tarball and compress it. Exclude files with the MPEG string in its file type.
      # -If the file type contains the string mpeg, $? (the exit status of the most recently executed command) expands to 0, 
      and the filename is redirected to the exclude option. Otherwise, it expands to 1.
      # -If $? equals 0, add the file to the list of files to be backed up.
      tar X <(for i in $DIR/*; do file $i | grep -i mpeg; if [ $? -eq 0 ]; then echo $i; fi;done) -cjf backupfile.tar.bz2 $DIR/*
      
      Reply
      • @Ravi,

        Thanks a lot for your time and further explanation.

        I understood him, the issue arise because I copied, pasted and ran the script but it gave me some errors.

        On examination, I found out the file was not pasted in its original format. I formatted it and it was fine.

        Thanks once more for your time; ravi means thrilled in French and I am thrilled.

        Reply
  2. Dear Gabriel Canepa
    Thank you for excelent material about LFCS exam. Just I would like to know. In your article you wrote that the compressed folder cannot be updated. I would like to know how about tar -rvf ?

    Reply
    • @Agil,
      Thank you for your question. Note that you can use tar -rvf to append files to an existing TAR file (.tar) but NOT to a compressed file (.tar.gz). If you try to add a file named foo to mycompressedfile.tar.gz as follows:
      tar -rvf mycompressedfile.tar.gz foo
      You’ll get the following error:
      tar: Cannot update compressed archives
      tar: Error is not recoverable: exiting now
      Hope it helps!

      Reply
  3. Hi Gabriel, thanks for this great course. I want to take the LFCS exam next Friday, Could you help me with some advices?. LFCS is more about your practice skills but I need to know by heart all commands?.

    Reply
    • @Erik,
      I am glad that you have found this series useful.
      The Linux Foundation just published a QA page about the certification programs http://t.co/1Oa9MwYpaQ. I would advice you to check it out.
      In addition, you should practice amd polish your command line skills until you feel comfortable. You can use different tools during the exam as long as they are in the Allowed list in the QA page mentioned above. So you can choose to perform a certain task using the resources you feel more comfortable with.
      Finally, you may want to check The Linux Command Line from linuxcommand.org.
      Hope it helps and good luck in the exam!

      Reply
  4. Hi Gabriel , I am studying with this excellent material for examination LFCS surrender next Thursday .
    Congratulations and thank you very much for sharing your knowledge with which we are beginning our way in Linux.
    Greetings from Buenos Aires, viva la patria !

    John.

    Reply
    • @Juan Ignacio,
      Thank you for your kind words about the LFCS series! Please note that I am currently writing another series covering the contents of the LFCE. You’re more than welcome to take a look!
      And best of luck with the exam next Thursday!
      Abrazo de gol.

      Reply

Got Something to Say? Join the Discussion...

Thank you for taking the time to share your thoughts with us. We appreciate your decision to leave a comment and value your contribution to the discussion. It's important to note that we moderate all comments in accordance with our comment policy to ensure a respectful and constructive conversation.

Rest assured that your email address will remain private and will not be published or shared with anyone. We prioritize the privacy and security of our users.