How to Archive Your Own Website Automatically
Patrick Bushe
August 24, 2024 · 5 min read
An archive of your own site protects you if your host fails, a redesign deletes pages, or you need proof of what your site said on a certain date. The best setup has two parts: public archived copies, and private backups you control.
Public copies in the Wayback Machine
- By hand: paste a page's address into Save Page Now at
web.archive.org/save. Wayback Quick Access adds a Save Page Now button to your toolbar, so archiving the page you're on takes one click. - Whole sections: with a free archive.org account, Save Page Now offers an option to save the page's outlinks too, which captures linked pages in one go.
- Automatically: the Internet Archive offers a Save Page Now API for account holders. A small scheduled script can read your sitemap and submit new or changed pages each week. Keep the pace gentle; the service has rate limits.
Public copies are dated and independent of you, which makes them useful as proof. See how Save Page Now works.
Private backups you control
- Your host's backups: check how often they run and how long they're kept. Many plans keep only a few days.
- Your platform's backup tools: on WordPress, a backup plugin can save files and the database on a schedule to cloud storage.
- Static copies: tools such as
wget --mirroror HTTrack download a browsable copy of your site to your computer. - Version control: if your site is built from code, a Git repository already records every change.
Store at least one backup somewhere other than your host. A backup on the same server disappears with it.
A simple schedule
- Every week: automatic backup of files and database.
- Every month: submit your main pages to Save Page Now, and download a copy of the backup.
- Before any redesign or migration: a full backup and a Save Page Now pass on every important page.
- Twice a year: test restoring a backup, so you know it works.
For organizations
Businesses and institutions that need complete, regular archives, for example for legal records, can use paid services. The Internet Archive's own Archive-It is used by libraries and universities.
If it's too late
If pages are already gone, see how to recover lost content from web archives.