How to Monitor Progress of (Copy/Backup/Compress) Data using ‘pv’ Command

When making backups, coping/moving large files on your Linux system, you may want to monitor the progress of an on going operation. Many terminal tools do not have the functionality to allow you to view progress information when a command is running in a pipe.

Read Also: Advance Copy Command to Show Progress Bar While Copying Files in Linux

Show Copy/Backup/Tar File Progress Bar in Linux

Show Copy/Backup/Tar File Progress Bar in Linux

In this article, we shall look at an important Linux/Unix command called pv.

Suggested Read: Progress – A Tiny Tool to Monitor Progress for (cp, mv, dd, tar, etc.) Commands

Pv is a terminal-based tool that allows you to monitor the progress of data that is being sent through a pipe. When using the pv command, it gives you a visual display of the following information:

  1. The time that has elapsed.
  2. The percentage completed including a progress bar.
  3. Shows current throughput rate.
  4. The total data transferred.
  5. and the ETA (estimated Time).

How to Install pv Command in Linux?

This command is not installed by default on most Linux distributions, therefore you can install it by following the steps below.

On Fedora, CentOS and RHEL

First you need to turn on EPEL repository and then run the following command.

# yum install pv
# dnf install pv            [On Fedora 22+ versions]
Sample Output
Dependencies Resolved

=================================================================================
 Package       Arch              Version                   Repository       Size
=================================================================================
Installing:
 pv            x86_64            1.4.6-1.el7               epel             47 k

Transaction Summary
=================================================================================
Install  1 Package

Total download size: 47 k
Installed size: 93 k
Is this ok [y/d/N]: y
Downloading packages:
pv-1.4.6-1.el7.x86_64.rpm                                 |  47 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : pv-1.4.6-1.el7.x86_64                                         1/1 
  Verifying  : pv-1.4.6-1.el7.x86_64                                         1/1 

Installed:
  pv.x86_64 0:1.4.6-1.el7                                                        

Complete!

On Debian, Ubuntu and Linux Mint

# apt-get install pv
Sample Output
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  pv
0 upgraded, 1 newly installed, 0 to remove and 533 not upgraded.
Need to get 33.7 kB of archives.
After this operation, 160 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/universe pv amd64 1.2.0-1 [33.7 kB]
Fetched 33.7 kB in 0s (48.9 kB/s)
Selecting previously unselected package pv.
(Reading database ... 216340 files and directories currently installed.)
Preparing to unpack .../archives/pv_1.2.0-1_amd64.deb ...
Unpacking pv (1.2.0-1) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up pv (1.2.0-1) ...

On Gentoo Linux

Use emerge package manager to install pv command as shown.

# emerge --ask sys-apps/pv

On FreeBSD Linux

You can use the port to install it as follows:

# cd /usr/ports/sysutils/pv/
# make install clean

OR add the binary package as follows:

# pkg_add -r pv

How Do I use pv Command in Linux?

pv is mostly used with other programs which lack the ability to monitor the progress of a an ongoing operation. You can use it, by placing it in a pipeline between two processes, with the appropriate options available.

The standard input of pv will be passed through to its standard output and progress (output) will be printed on standard error. It has a similar behavior as the cat command in Linux.

The syntax of pv command as follows:

pv file
pv options file
pv file > filename.out
pv options | command > filename.out
comand1 | pv | command2 

The options used with pv are divided into three categories, display switches, output modifiers and general options.

Some options under display modifiers.
  1. To turn on the display bar, use the -p option.
  2. To view the elapsed time, use the –timer option.
  3. To turn on ETA timer which tries to guess how long it will take before completion of an operation, use the –eta option. The guess is based on previous transfer rates and the total data size.
  4. To turn on a rate counter use the –rate option.
  5. To display the total amount of data transferred so far, use the –bytes option.
  6. To display progress inform of integer percentage instead of visual indication, use the -n option. This can be good when using pv with the dialog command to show progress in a dialog box.
Some options under output modifiers.
  1. To wait until the first byte is transferred before displaying progress information, use the –wait option.
  2. To assume the total amount of data to be transferred is SIZE bytes when computing percentage and ETA, use –size SIZE option.
  3. To specify seconds between updates, use the –interval SECONDS option.
  4. Use –force option to force an operation. This option forces pv to display visuals when standard error is not a terminal.
  5. The general options are –help to display usage information and –version to display version information.

Use pv Command with Examples

1. When no option is included, pv commands run with default -p, -t, -e, -r and -b options.

For example, to copy the opensuse.vdi file to /tmp/opensuse.vdi, run this command and watch the progress bar in screencast.

# pv opensuse.vdi > /tmp/opensuse.vdi

Copy Files Show Progress Bar

2. To make a zip file from your /var/log/syslog file, run the following command.

# pv /var/log/syslog | zip > syslog.zip

Zip File Show Progress Bar

3. To count the number of lines, word and bytes in the /etc/hosts file while showing progress bar only, run this command below.

# pv -p /etc/hosts | wc

Count Lines, Words and Bytes in Linux

4. Monitor the progress of creating a backup file using tar utility.

# tar -czf - ./Downloads/ | (pv -p --timer --rate --bytes > backup.tgz)

Create Tar Backup: Show Progress Bar

5. Using pv and dialog terminal-based tool together to create a dialog progress bar as follows.

# tar -czf - ./Documents/ | (pv -n > backup.tgz) 2>&1 | dialog --gauge "Progress" 10 70
Create Dialog Progress Box

Create Dialog Progress Box

Summary

This is a good terminal-based tool that you can use with tools that do not have the ability, to monitor the progress of an operations such as coping/moving/backing up files, for more options check man pv.

I hope you find this article helpful and you can post a comment if you have any ideas to add about using pv command. And if you get any errors while using it, you can as well leave a comment.

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

Support Us

We are thankful for your never ending support.

Aaron Kili

Aaron Kili is a Linux and F.O.S.S enthusiast, an upcoming Linux SysAdmin, web developer, and currently a content creator for TecMint who loves working with computers and strongly believes in sharing knowledge.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

24 Responses

  1. kk says:

    Is there any reason why pv might slow things down? See example below:

    time pigz -dk MYSQL_BACKUP_2018-06-08-17\:26\:15.sql.gz | mysql -pmetatest
    Warning: Using a password on the command line interface can be insecure.

    real 0m29.868s
    user 0m5.108s
    sys 0m2.020s

    time pv MYSQL_BACKUP_2018-06-08-17\:26\:15.sql.gz | pigz -d | mysql -pmetatest
    Warning: Using a password on the command line interface can be insecure.
    215MiB 0:01:16 [2.81MiB/s] [========================================================================>] 100%

    real 1m16.837s
    user 0m13.868s
    sys 0m2.392s

    • Aaron Kili says:

      @kk

      This is quite interesting! We have to investigate this further, to come up with a solid conclusion. Many thanks for sharing this concern.

      • xrkr says:

        Hello Aaron/Kk,

        Was there any conclusion? We observed a similar lag while using pv command to transfer the files over the p2p network. The pv command didn’t return to the command prompt quickly while over the LAN it was quick.

  2. Ronald Trumpet says:

    >> On FreeBSD Linux

    Nice joke.

  3. gosho says:

    Formating hard disk with dd command using pv:

    # dd if=/dev/zero | pv -pbtr -s 120033041920 | dd of=/dev/sdX
    

    note: 120033041920 bytes is exact size of the disk, get your with fdisk -l
    sdX is the disk you want to dd, change X with your letter- a, b, c…

    • Ravi Saive says:

      @Gosho,

      Thanks for the useful tip about dd command with pv, sounds great, will create a short article on same. Else, you can also create a short 300 words article on same, if you want we will publish the article under you name in Tecmint..

  4. Yoander says:

    please could you tell us what is the meaning of: -t, -e, -r and -b options?

  5. Alagirisamy says:

    I am getting the below error while copying 4 Gb files from workstation to pendrive

    [[email protected] home]# ls
    Alagiri CentOS-7-x86_64-DVD-1602-01.iso Dvois lost+found
    [[email protected] home]# pv CentOS-7-x86_64-DVD-1602-01.iso > /media/ammu/CentOS-7-x86_64-DVD-1602-01.iso
    pv: write failed: File too large=================================================================================> ] 97% ETA 0:00:26
    You have new mail in /var/spool/mail/root

  6. R says:

    New version of coreutils (8.24) adding a status progress to dd tool:

    Usage on Xubuntu 15.10:

    Open terminal shell and type these commands:

    wget ftp://ftp.gnu.org/pub/gnu/coreutils/coreutils-8.24.tar.xz
    tar -xf coreutils-8.24.tar.xz
    cd coreutils-8.24
    ./configure && make -j $(nproc)

    Run dd as root:

    sudo ./dd if=/dev/sdc of=/dev/sda conv=noerror status=progress

    You will see: Bytes, Seconds and Velocity (Bytes/seconds)

    To check versions of dd:

    Native:

    dd –version

    New (cd coreutils-8.24/src):

    ./dd –version

  7. Tobi says:

    Check the network speed between two computers with ‘netcat’ and without utilizing a harddisk:
    At the first computer do a:
    netcat -l -p 1234 | pv > /dev/null
    Then go to the second computer and type:
    cat /dev/zero | pv | netcat ip_of_first_computer 1234
    Helped me to identify slow network switches.

  8. Emiliano says:

    Another GREAT use of “pv” is using it with “dd”.
    works great for me.

    dd if=/dev/sda1 | pv |dd of=/home/user/backup.iso

    Definitely it’s a “must” for “dd”. This process always take a long time to finish.

    Great article.

  9. Josmell Jimenez says:

    Excellent as always, great and easy explanation for every newbie and advanced linux user.

    Keep it up!

  10. Jon Loveless says:

    excellent tutorial on pv. I loved the video examples. It just doesn’t any more clear than that! thanks.

Got something to say? Join the discussion.

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.