Last Updated: Sunday 3rd November 2013

In the last article, titled How to Recursively Copy a Folder (Directory) in Python, I covered how to copy a folder recursively from one place to another.

As useful as that is, there is still something else that we could add that would make copying a directory much more user friendly!

Enter the progress bar from stage left. It's a useful utility that all big name software uses to tell the user something is happening and where it is with the task it is doing.

So how do we incorporate this into copying a directory? Glad you asked, my question filled friend!

The Progress Bar

First, we'll make a simple class to make printing progress 100 times easier as well as give us a nice reusable class to use in other projects that might need it.

Here we go. First we'll add the __init__ method:

That wasn't too hard! We pass in the message we want to print, the width of the progress bar, and the two symbols for progress done and empty progress. We check that the width is always greater that or equal to zero, because we can't have a negative length! :)

Simple stuff.

Alright, on to the hard part. We have to figure out how to update and render the progress bar to the output window without printing a new line every single time.

That's where the carriage return (\r) comes in. The carriage return is a special character that allows the printed message to start at the beginning of a line when printed. Every time we print something, we'll just use it at the beginning of our line and it should take care of our needs just fine. The only limitation this has is that if the progress bar happens to wrap in the terminal, the carriage return will not work as expected and will just output a new line.

Here is our update function that will print the updated progress:

What we just did is calculate the number of filled blocks using the total blocks needed. It looks a little complicated, so let me break it down. We want the number of filled blocks to be the progress divided by 100% divided by the total number of blocks. The calculateAndUpdate function is just to make our life a little easier. All it does is calculate and print the percentage done given the current number of items and the total number of items.

That call to float is there to make sure the calculation is done with floating point precision, and then we round up the floating point number with round and make it an integer with int.

Then the empty blocks are simply the totalBlocks minus the filledBlocks.

We take the number of filled blocks and multiply it by the progressSymbol and add it to the emptySymbol multiplied by the number of empty blocks.

We use the carriage return at the beginning of the message to set the print position to the beginning of the line and make a message formatted like: "Message! ▣ ▣ ▣ □ □ □ 50%".

And, finally, we print the message to the screen. Why don't we just use the print function? Because the print function adds a new line to the end of our message and that's not what we want. We want it printed just as we set it.

On with the show! To the copying a directory function!

Copying a Directory (Folder) with Progress in Python

In order to know our progress, we'll have to sacrifice some speed. We need to count all of the files that we're copying to get the total amount of progress left, and this requires us to recursively search the directory and count all of the files.

Let's write a function to do that.

Pretty straight forward. We just checked if the directory passed in is a directory, then if it is, recursively walk the directories and add the files in the directories to the files list. Then we just return the length of the list. Simple.

Next is the copying directories function. If you looked at the article mentioned at the top of this article, you'll notice that I used the shutil.copytree function. Here we can't do that because shutil doesn't support progress updates, so we'll have to write our own copying function.

Let's do it!

First we create an instance of our ProgressBar class.

Next, we define a function to make directories that don't exist yet. This will allow us to create the directory structure of the source directory.

If the directory doesn't exist, we create it.

Now for the copy function. This one is a bit of a doozy, so I'll put it all down in one place, then break it down.

Firstly, we count the number of files.

Nothing too difficult about that.

Next, if there are actually files to copy, we create the destination folder if it doesn't exist and initialize a variable to count the current number of files copied.

Then, we walk the directory tree and create all the directories needed.

The only tricky part here is that I replaced the source directory string in path, which is the root path, with the destination folder path.

Next, we copy files and update progress!

Here, we go through all the files, copy them, update the number of files copied so far, and then draw our progress bar.

That's it! We're done! Now you have a nice copying function that will alert you of the progress while copying files.

Moving a Directory (Folder) with Progress in Python

As a note, you may also change the shutil.copy function call in copyFilesWithProgress to shutil.move and have a move progress bar instead. All the other code would be the same, so I'm not going to rewrite it here. I'll leave that up to you! :)

Until the next time, I bid you adieu.

About The Author

  • RunLevelZero

    This is great but I am not experienced and I can’t get this to output anything :). I am sure I am missing the src and dest. With the code so broken up it’s hard for me to puzzle it back together again. Would you be able to break this down so I know where the src and dest dir needs to exist in the code? I’m sorry for such a simple question but I just don’t see where I need to input this data. Is it taking it from the previous article on how to copy files and folders? I want to copy one directory with files in it to a folder on several remote servers with a progress bar and it would be nice to verify the copy process as well. TIA.

    • http://jacksonc.com Jackson Cooper

      No worries :). I’ve uploaded the source code as a zip file. There’s two files, the one you should run is copy_dir.py. You’ll need to change the directories you want to copy at the bottom of copy_dir.py. Keep in mind that this article is just to learn Python, and probably isn’t a good idea to use in practice, without some modifications or further development! Also, it updates the progress per file transferred, not per bit/byte. So if there’s 4 files, the first 3 are 5 KB, and the third is 100 MB, after transferring 3 files it will show 75% as the progress.

      To work over a network is a whole different ball game. If you’re running Linux / Mac / anything POSIX, I’d suggest using rsync</code≥ with the --progress option. It does delta transmission too (only moves the changes). Windows... I have no idea :-/. It'd be cool to see it be done with Python though!

      • Jimmy Xaime Kinglay

        Hi there! First, really cool script – your thing not only showed my some way of resolving my issue but also, teached me a lot of new things in python. I’m a newbie, trying to code since about couple of weeks so it took some time for me to really understand how does this thing work and it’s brilliant :)

        I also decided to try to make it more accurate in my case – I need to use such a progress bar to copy folder with 4 files (3 of them about 1KB and one pretty big, 25GB). As you mentioned above (your example totally fits my case), copying first 3 files will result with progress bar showing 75% which is not really what I needed. I found my own way to count size of copied files, not number of them – did so using os.path.getsize. I’ve modified countFiles function and copyFilesWithProgress to count weight, and result worked fine but with one mistake on my side – size (or number) of already copied files updates itself after copying every single file. Soo it looks like my progress bar is showing 1% for a long long time and then, suddenly it gets 100% (when that 25GB fatty boy is done).

        My version is not lying to me but it’s not telling all the truth neither so it’s almost good but not enough for me :) Are you able to show me some way to void that calculateAndUpdate function not only when the file is copied, but during that copying process? Is that even possible?

        Regards!
        Jimmy

  • Vivek Singh

    Thanks for such a great article, but i ‘m facing the same issue as “RunLevelZero” said. could you please point me over to the source code download.