Python 3 & Wget – Download Images from CSV.

2018, Nov 12    

Objective

I wanted to download a list of files from an S3 bucket using python 3. I used a standard CSV in UTF-8 format. Here is an example CSV. It’s one column with just the filenames. 

I could have built out the full url in the CSV, but I decided to just do it in the code.  Below is the full python script. 

import csv
import wget
#open import file
with open("c:\\csv\\files-to-download.csv", newline='', encoding='utf-8') as f:
#Assign the import file to the DictReader "reader"    
    reader = csv.DictReader(f)
    #Now loop through all rows and build out variables    
    for row in reader:
        filename = row['name']
        print('Beginning file download with wget module')
        url = 'http://www.example.com/images/'
        wget.download(url + filename, 'c:\\csv\\images\\' + filename) 

Breakdown

First, I importe csv, and **wget**. If you don’t have [wget](https://pypi.org/project/wget/) module you can install it by using pip. 

pip install wget

Next, I opened the CSV that contains the list of files I needed to download. 

with open("c:\\csv\\files-to-download.csv", newline='', encoding='utf-8') as f:

After that I passed the file into a DictReader called “Reader”: 

reader = csv.DictReader(f)

Now, I will loop through each row in the file

for row in reader:

On the first pass through the rows I want to assign the row with the column header “name” to the variable called filename

filename = row['name']

Optional –  Print a message that the program is about to download the file. 

print('Beginning file download with wget module')

To build the full path download url, I will assign the base url to the variable called url. 

 url = 'http://www.example.com/images/'

Here at the end the wget.download takes two different arguments. The url of the file that will be downloaded and the local path file destination.  You can see here we are building the full path with url + filename.  For the local path, I chose to build out the local destination with a string and a variable: ‘c:\csv\images\’ + filename

wget.download(url + filename, 'c:\\csv\\images\\' + filename)

Here is the code in action downloading our 4 test pictures. 

</figure>