The 2004 Perl Advent Calendar
[about] | [archives] | [contact] | [home]

On the 7th day of Advent my True Language brought to me..
Term::ProgressBar

Some things take a long time - no matter how fast our computers get there's just no escaping the fact that sometimes things take more than a few seconds to get done. When this happens it's nice to give the user some kind of feedback about how long the program will take to complete the task - if the task is progressing at all.

Term::ProgressBar, as the name implies, can be used to draw a progress bar in your terminal. It will also handily handle the maths involved in updating the bar - which while not complicated, means that it's trivial to use...meaning that there's no excuse for not adding one to a long running process.

Let's look at the simple case of downloading a collection of pages from the web.

  #!/usr/bin/perl
  # turn on perl's safety features
  use strict;
  use warnings;
  # load the modules I need to know
  use Term::ProgressBar;
  use LWP::Simple qw(mirror);
  # work out what pages we're getting
  my @users = qw( 2shortplanks acme pudge torgox mschwern );
  # create a new progress bar
  my $no_pages = @users;
  my $progress = Term::ProgressBar->new({count => $no_pages});
  # get all the pages
  my $got = 0;
  foreach my $user (@users)
  {
    # work out the url
    my $url = "http://use.perl.org/~".$user."/journal/rss";
    # download the url and save it to disk
    mirror( $url, "${user}.rss" );
    # update the progress bar
    $progress->update(++$got);
  }

This creates a bar on the terminal that looks like this:

    0% [                                                            ]

Which slowly updates

   60% [====================================                        ]

Until it's done

  100% [============================================================]

By passing extra options to the constructor we can ask it to estimate how long is left in the process based on how long the previous stages took:

  my $progress = Term::ProgressBar->new({
    count => $no_pages,
    ETA   => "linear", 
  });

This gives us a much more useful progress bar:

   40% [===================                               ]0m07s Left

We can also name our progress bars:

  my $progress = Term::ProgressBar->new({
    count => $no_pages,
    ETA   => "linear",
    name  => "downloading",
  });

This places a name next to the progress bar on the left

  downloading:  40% [===============                      ]0m07s Left

Printing Messages

If we want to provide more info to the screen, we might try printing out the url that we're currently downloading:

  # get all the pages
  my $got = 0;
  foreach my $user (@users)
  {
    # work out the url, print it out
    my $url = "http://use.perl.org/~".$user."/journal/rss";
    print "fetching $url\n";
    # download the url and save it to disk
    mirror( $url, "${user}.rss" );
    # update the progress bar
    $progress->update(++$got);
  }

However, there's a problem with this. Whenever the progress bar needs to be updated Terminal::ProgressBar has to sent control characters to the terminal to move the cursor back to the left of the screen so the new progress bar it's printing out covers up the old terminal bar. If we print anything out this causes the terminal to scroll and the old progress bar moves up the terminal:

  downloading:   0% [                                      ]
  fetching http://use.perl.org/~2shortplanks/journal/rss
  downloading:  20% [=======                               ]0m12s Left
  fetching http://use.perl.org/~acme/journal/rss
  downloading:  40% [===============                       ]0m08s Left
  fetching http://use.perl.org/~pudge/journal/rss
  downloading:  60% [======================                ]0m05s Left
  fetching http://use.perl.org/~torgox/journal/rss
  downloading:  80% [==============================        ]0m02s Left
  fetching http://use.perl.org/~mschwern/journal/rss
  downloading: 100% [======================================]-- DONE --

How ugly! (Actually, it's even worse than that - the line wraps don't happen nearly as nicely as I've put above.) For this reason Terminal::ProgressBar provides the message function that can be used in place of printing output to the terminal:

  # get all the pages
  my $got = 0;
  foreach my $user (@users)
  {
    # work out the url, print it out
    my $url = "http://use.perl.org/~".$user."/journal/rss";
    $progress->message("fetching $url\n");
    # download the url and save it to disk
    mirror( $url, "${user}.rss" );
    # update the progress bar
    $progress->update(++$got);
  }

message removes the old progress bar, prints the message, and then redraws the progress bar again. This has the effect of slowly printing the debug text while leaving the progress bar as the last thing on the display:

  fetching http://use.perl.org/~2shortplanks/journal/rss              
  fetching http://use.perl.org/~acme/journal/rss                      
  fetching http://use.perl.org/~pudge/journal/rss                     
  downloading:  40% [===============                       ]0m07s Left

Frequently Updating Terminals

Printing things out to terminals is fast - but not that fast. If we're connected to the terminal over a slow link (for example, sshed in over a GPRS modem) then the time taken to print to the terminal may start being a significant time of the run. Even the fast terminals on our whizzy laptops may not be able to keep up if the time taken to run something is quick.

Let's write an example that runs though a thousand iterations without doing anything time consuming.

  #!/usr/bin/perl
  # turn on perl's safety features
  use strict;
  use warnings;
  use Term::ProgressBar;
  my $progress = Term::ProgressBar->new({
    count => 10_000,
    name  => "test",
  });
  my $got = 0;
  for (1..10_000)
  {
    ++$got;
    $progress->update($got);
  }

Running this on my laptop takes about nine seconds:

  bash$ time perl doit
  test: 100% [=======================================================]
  real    0m9.448s
  user    0m3.860s
  sys     0m0.200s
  bash$

Commenting out the update statement:

  my $got = 0;
  for (1..10_000)
  {
    ++$got;
    # $progress->update($got);
  }

And running it again gives very different results:

  bash$ time perl doit
  test:   0% [*                                            ]
  real    0m0.243s
  user    0m0.170s
  sys     0m0.030s
  bash$

So, what can we do to improve the situation? Well, we really only need to update the progress bar when it's time to draw the next = on the screen. Since the progress bar object knows how wide the terminal is (68 chars in the examples above), and how many steps there are in total (ten thousand) it can work out at what point after the current update an update to the screen is actually needed. It returns that number from the <update> method so we can modify the code to only call update again when you reach that number:

  my $got = 0;
  my $needed = 0;
  for (1..10_000)
  {
    ++$got;
    $needed = $progress->update($got)
       if $got >= $needed;
  }

Which is much quicker:

  bash$ time perl doit
  test: 100% [=======================================================]
  real    0m4.686s
  user    0m1.310s
  sys     0m0.070s
  bash$ 

  • LWP::Simple
  • LWP::Parallel - download much more efficiently than explained above