Web TechnologiesCourse and practice

No way back

Why a backup counts only once someone has restored it, why it must be kept apart from what it protects, and why every risky change needs a way back before it starts.

5 parts · about 7 min

Present Cheat sheet

A backup nobody has restored is only a hope

A backup counts only once someone has restored it and checked that everything came back.

Ask any team whether they have backups, and they'll say yes. Ask when they last restored one, and often nobody knows. A backup is a copy of the data, kept for the day the data is lost. It only proves that it works on the day you put it back. That's a restore.

Backups fail silently. Take a wedding photographer whose site holds a private gallery for each couple. The hosting made a backup every night. When a plugin update deleted the galleries, the photographer asked for a restore, and learned three things. The backup held the pages, but not the photos, which were stored on another service. The newest copy that worked was four months old. Since a disk had become full, every nightly backup had been saved as an empty file, and nobody was told. And nobody knew how to put it back, so it took four days.

None of this shows until you try. So a backup counts only once someone has restored it, on a copy rather than on the live site, and checked that everything came back. This is a restore test. Ask when the last one happened, how long it took, and what was missing. Then repeat it every few months, because what the site stores keeps changing.

The lesson What keeps running asked this as one question at the handover. This lesson looks at what lies behind it.

The example below shows the photographer's backups twice. On the left is what the hosting's backup page said before the problem: everything looked fine. On the right is what a restore test would have shown, months earlier.

What the backup page said

Code
The hosting's backup page
Backups: on
How often: every night
Last backup: successful
Kept for: 6 months

What a restore test showed

Code
Restore test, 12 March
Restored on: a test copy
Pages: all back
Photos: missing, because they
  are stored on another service
Newest copy that works:
  4 months old. The newer ones
  are empty, since the disk
  became full.
Time to restore: 4 days
Next test: June
Example 1A photographer's backups: what the backup page said, and what a restore test would have shown

Cheat sheet

Backupsauvegarde
a copy of a site's data kept in a safe place, to recover it after a loss or an error.
Restorerestaurer
to replace the current data with a backup, to recover lost data or to check that the backup works.
Restore testtest de restauration
putting a backup back on a copy of the site, to prove that everything returns and to measure how long it takes.
Pluginplugin, extension
an add-on, often made by another company, that gives a tool such as WordPress a new feature.
Hostinghébergement
the service that stores a site's files and sends them to visitors.

Kept apart from what it protects

A backup protects you only from what it doesn't share with the original. A copy kept next to the original is lost with it.

Think of a sports club that keeps its member database on the treasurer's laptop, and copies it every week to a USB drive. The drive travels in the same bag. When the bag is stolen on a train, both are gone. A copy on the same server has the same weakness: a disk failure, an attack or a mistake that deletes the server takes the copy too. In March 2021, a fire destroyed one of OVHcloud's data centres in Strasbourg. Some customers lost their sites for good, because their backups were in the same place.

Accounts work the same way. If every copy sits with the same provider, in the same account, then a hacked, closed or unpaid account takes them all.

Attackers know this. Ransomware locks an organisation's data and asks for money to unlock it, and the attackers search for the backups first. So at least one copy must be out of their reach: with another provider, in another account, or offline.

Two common tools are not backups at all. A synced folder, such as Dropbox or Google Drive, copies every change to every device, so a deleted or locked file is deleted or locked everywhere. And an online tool that keeps your content on its servers protects its own disks. It doesn't protect you from your own mistakes, or from losing the account.

A common rule sums this up: 3-2-1. Keep three copies of the data, on two different kinds of storage, with one of them in another place.

Where the backup is kept decides which incidents it survives1. The same server: A disk failure, an attack or a deleted server loses both copies. Like a laptop and its USB drive in the same bag. 2. The same account, at one provider: A closed, unpaid or hacked account loses both copies. Two servers do not help if one account holds them. 3. Your provider and Another provider, or offline: One incident can't reach both copies. Ransomware can't lock this copy either.The same serverThe live dataWhat visitors useThe backupA copy to recover lostdataA disk failure, an attack or adeleted server loses both copiesLike a laptop and its USB drive in the samebagThe same account, at one providerThe live dataWhat visitors useThe backupA copy to recover lostdataA closed, unpaid or hacked accountloses both copiesTwo servers do not help if one account holdsthemYour providerAnother provider, or offlineThe live dataWhat visitors useThe backupA copy to recover lostdataOne incident can't reach bothcopiesRansomware can't lock this copy either

Swipe sideways to see the whole figure.

Figure 1

Cheat sheet

Hosting providerhébergeur
the company that rents you the servers where a site is stored, such as OVH or Netlify.
Data centrecentre de données, datacenter
a building full of servers, where hosting providers keep their customers' sites and data.
Ransomwarerançongiciel
an attack that locks an organisation's data and asks for money to unlock it.
Syncsynchronisation
a service that keeps the same files on several devices, so that a change or a deletion on one reaches all of them.
3-2-1 rulerègle 3-2-1
keep three copies of the data, on two kinds of storage, with one copy in another place.
Backupsauvegarde
a copy of a site's data kept in a safe place, to recover it after a loss or an error.

How far back, and how often

Problems are rarely noticed on the day they start. A wrong import, a deleted folder or an attack can stay unnoticed for days or weeks. So the question isn't only "do we have backups?" but also "how far back do they go?".

Take a music school's website. A cleaning tool deletes the teachers' photos by mistake. Nobody looks at the teachers' page until a parent asks, sixteen days later. The host keeps one backup a day, for seven days. Every backup that still exists was made after the mistake, so the photos are gone. With a month of backups, they would have come back in minutes.

So ask how many days of backups are kept, and compare that with how long a problem could stay unnoticed.

Then ask how often a backup is made. That depends on how much data the organisation can afford to lose. For a site whose content changes once a month, a weekly backup loses little. For a shop that takes orders all day, a backup every night means losing up to a day of orders. Is that acceptable, or must it be every hour? This is a business decision, so ask the manager, not only the developer. Teams call the answer the RPO.

Last, make sure that someone receives an alert when a backup fails. Otherwise, a backup that stopped in March may stay unnoticed until the day you need it.

Backups kept for seven days can't undo a mistake noticed sixteen days later1. Day 3: a cleaning tool deletes the teachers' photos. 2. Day 19: a parent asks where the photos are. 3. Only seven backups are left, and all of them were made after the mistake. 4. A month of backups would still hold day 2, the day before the mistake.Each square is the backup made on the night of that day1234567891011121314151617181920Day 3: a cleaning tool deletesthe teachers' photosDay 19: a parent asks wherethe photos areA dashed square is a backup already deletedOnly 7 backups are left, all made after the mistakeA month of backups would still hold day 2

Swipe sideways to see the whole figure.

Figure 2

Cheat sheet

Backup retentiondurée de rétention des sauvegardes
how many days or months of backups are kept before the oldest ones are deleted.
RPOPDMA, perte de données maximale admissible
how much recent data an organisation can afford to lose, such as one day of orders, which sets how often to make a backup.
Backupsauvegarde
a copy of a site's data kept in a safe place, to recover it after a loss or an error.
Restorerestaurer
to replace the current data with a backup, to recover lost data or to check that the backup works.
Ransomwarerançongiciel
an attack that locks an organisation's data and asks for money to unlock it.

Every risky change needs a way back

Many data losses happen during a planned change, not an attack. Switching to a new payment provider, changing the site's address, running a script over every account, deleting "old" records with one click: each of these can go wrong, and some can't be undone.

Take a city library that wants to merge 4,000 duplicate reader accounts with one script, the evening before the university term starts, when thousands of students will log in. Before you say yes, ask for two things.

First, a fresh backup, made just before the change and checked, not last week's. Without it, a failed merge loses every loan and reservation since the last good copy.

Second, a way back. If it goes wrong, what exactly do we do, who decides, and how long does it take? Agree in advance on what "it went wrong" looks like, such as readers who can't log in. Then the team undoes the change, instead of trying to fix it on the live site for hours.

Then make the risk smaller. Try the change where it can't hurt: on a copy of the data, often called staging, and then on a small part first. Merge 50 accounts, check them, then do the rest. Choose a quiet day, when people are around to watch and fix it, not the evening before the busiest week of the year. And right after the change, check what matters most. Here, that's logging in, borrowing and reserving.

For your portfolio, the way back is simpler. Before the agent rebuilds a whole page, make sure the last version is saved as a commit, and that the host keeps the previous version online. The lesson Following the work shows how.

Before a risky change, prepare a backup, a way back and a trial; on the day, check the result or undo it1. A fresh backup 2. A way back 3. A trial first 4. The change 5. Check it works 6. Undo itBefore the changeOn the dayA fresh backupMade just before thechange, and checkedA way backHow to undo it, whodecides, how longA trial firstOn a copy, then on a smallpartThe changeOn a quiet day, withpeople aroundCheck it worksLogging in, forms, paymentand searchUndo itIf it went wrong, asagreed in advance

Swipe sideways to see the whole figure.

Figure 3

Cheat sheet

Backupsauvegarde
a copy of a site's data kept in a safe place, to recover it after a loss or an error.
Rollback planplan de retour arrière
what the team will do to undo an update if it goes wrong, and how long it will take.
Stagingpréproduction, recette
a test copy of a site that works like the live site, where changes are tried before they go live.
Smoke testtest de fumée, smoke test
a quick check of a site's main paths right after an update, such as browsing, the basket and payment.
Commitcommit
a saved version of the project, with a short message that says what changed and why.

What to ask

Ask these questions when you become responsible for a site, before any risky change, and again every few months.

When someone tells you that backups exist, ask when one was last restored, how long it took, and whether everything came back: the pages, the database, the photos and the settings. If nobody knows, ask for a restore test on a copy, with a date. Until a restore has worked, the backups are only a hope.

Ask where each copy is: on the same server, with the same provider, in the same account, or somewhere else. Ask how far back the backups go, and how often they run. Then ask the manager how much data the organisation can afford to lose, a day or an hour, and check that the backups match. Ask who receives the alert when a backup fails.

Before any risky change, ask for a fresh backup, a written way back, and a trial on a copy or on a small part. Ask who checks the important paths right after the change, and who decides to undo it.

For your portfolio, your code is on GitHub, and the host keeps old versions. But where else are your photos, your CV and your form's messages? Keep a copy of your content folder somewhere other than your laptop.

Cheat sheet

Restore testtest de restauration
putting a backup back on a copy of the site, to prove that everything returns and to measure how long it takes.
Backup retentiondurée de rétention des sauvegardes
how many days or months of backups are kept before the oldest ones are deleted.
RPOPDMA, perte de données maximale admissible
how much recent data an organisation can afford to lose, such as one day of orders, which sets how often to make a backup.
3-2-1 rulerègle 3-2-1
keep three copies of the data, on two kinds of storage, with one copy in another place.
Rollback planplan de retour arrière
what the team will do to undo an update if it goes wrong, and how long it will take.

Practise this strand

These questions put you in the situations this lesson describes. You answer in your own words, and you get a level, feedback and a full answer.