This lesson will allow you to overcome such an obstacle. Reports, you can see the limits to the current means of presentation If you want several decades’ worth of annual The only option is to right click –> save as each image to aĭirectory on your computer. Moreover, if you want the document for offline viewing, This allows you to see the original document, but it isĪlso cumbersome because it requires you to scroll through each Report” link, which will take the user to a page with LAC’s embedded Of each page, or click on the “View a scanned page of original The current system allows a user the option to read a plaintext version You wanted to download an entire report, or reports for several decades. Take for example the Indian Affairs Annual Reportsĭatabase hosted on the Library and Archives Canada website. To access hitherto unavailable or cost-prohibitive materials, they canĪlso be limited by the manner in which content is presented and

In other words, while online collections often allow historians Increased accessibility does not always translate into increased However, we will focus on Wget and building your Python skills.Īrchival websites offer a wealth of resources to historians, but Performing automated downloads from the command line. Similar functions to the ones demonstrated in this lesson canīe achieved using curl, an open-source software capable of Script, and will also introduce you to a few of Wget’s other usefulįeatures.

Teach you how to parse and generate a list of URLs using a simple Python Websites with assistance from the Python programing language. The following tutorial provides three examples of how Wget canīe used to download large collections of documents from archival Lessons that focus on other uses for Wget’s recursive retrievalįunction. Line, it’s time to expand your web-scraping skills through a few more Specific files from websites like ActiveHistory.ca via the command Now that you have learned how Wget can be used to mirror or download

More Complicated Recursive Retrieval: A Python Script for Leading Zeros.

Recursive Retrieval and Wget’s ‘Accept’ (-A) Function.

A Second Example: The National Archives of Australia.

Recursive Retrieval and Sequential URLs: The Library and Archives Canada Example.

Wget download file