Backup and restore

This page is for administrators. It explains what to save, how to take a backup by hand or every night automatically, how to prove that a backup can be restored, and how to restore an installation or return to the previous version after a failed update.

A backup is only worth something once you have restored it. Plan a restore test from the start.

What to save

The database and the books volume belong together: always restore both from the same backup.

To save a single book with all its work, export it as a project archive instead (Export › Complete project (.zip)) and restore it from the library (Add content › Restore a Libris archive). A whole series — its volumes, glossary, identities and their links, Series Bible and settings — is exported in one file with Export the series on the series page and restored with Add content › Restore a Libris archive › A whole series (format in architecture.md). Neither replaces a full backup: accounts, providers, keys and shared glossaries are not in them.

Back up by hand

Run these commands in the Libris directory. They stop the web application and the worker for the time of the copy, so that the database and the files match.

mkdir -p backups && chmod 700 backups
docker compose stop api worker
docker compose exec -T database sh -c 'pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc' > backups/database.dump
docker compose run --rm --no-deps -T api tar -C /data -czf - . > backups/books.tar.gz
cp .env backups/config.env
chmod 600 backups/*
docker compose up -d --no-build --wait

Copy the backups directory to another machine. A backup kept on the same disk does not survive the loss of that disk.

On a host driven by the production deployment script, librisctl backup takes the database dump alone (without stopping anything) and prints its path; the books volume and the configuration are covered by the scheduled backup below.

Back up every night

backup/libris-backup makes a verified backup, freezing the API and the worker for the length of the dump and the archive (usually seconds to a few minutes), writes it to a share mounted from another machine, and removes old backups. A systemd timer runs it every night. Nothing is installed automatically.

Install it

As root on the Docker host, from the registry installation directory (default /opt/libris):

install -m 0755 backup/libris-backup backup/libris-restore /usr/local/sbin/
install -m 0644 backup/libris-backup.service backup/libris-backup.timer /etc/systemd/system/
install -m 0600 backup/libris-backup.conf.example /etc/libris-backup.conf
editor /etc/libris-backup.conf
systemctl daemon-reload

1. Mount a share from another machine (NFS, SMB, sshfs…) permanently, for example at /mnt/libris-backup, and create a destination directory in it. Root must be able to write there, file permissions must be kept (backups are created 0600), and it needs room for as many backups as you retain.

2. Edit /etc/libris-backup.conf (see the table below). Set LIBRIS_BACKUP_MOUNTPOINT so that the backup fails instead of silently filling the local disk when the share is not mounted.

3. Run a first backup and read its result:

   systemctl start libris-backup.service
   journalctl -u libris-backup.service -n 20
   

4. Enable the nightly run (03:30, with up to 20 minutes of random delay; a run missed while the machine was off happens at the next start):

   systemctl enable --now libris-backup.timer
   systemctl list-timers libris-backup.timer
   

5. Watch for failures: check systemctl is-failed libris-backup.service in your monitoring, or add an OnFailure= drop-in that calls your usual notification unit.

Settings

When .env is not included, save it yourself somewhere safe: it rarely changes.

What a backup contains

The dump and the archive are taken at different instants, and the API and the worker add, replace and delete files (EPUBs, sources, covers, exports) after committing their rows. So the script runs docker pause on the api and worker containers before the dump and unpauses them as soon as the archive is written (also when it fails): requests wait, nothing is refused or lost, and no file changes between the two phases. It lists the volume (path, size, modification time) before the dump and after the archive; if any file was added, deleted or replaced in between, the backup fails with files of … changed while the backup ran, keeps nothing and never prints “written and verified”. Both files are then read back entirely, and the hash of each archived file is written to books.sha256; the archive must list exactly the files the volume held.

Limits: the freeze covers the two containers of the Compose project, not a librisctl command or another container writing to the volume (the volume check catches those, and the run is simply repeated); the check compares size and modification time, not content, so a file rewritten with the same size and timestamp goes unseen; with LIBRIS_BACKUP_FREEZE=false a backup on a busy installation fails whenever a file changes during the run. Paused containers keep their database connections and their health checks stall: a long pause may show them as unhealthy until they resume. A backup before an upgrade should still stop the services first. The backup is written to a hidden .libris-….partial directory and renamed only at the end, so an interrupted run never looks like a backup. Old backups are removed only after a successful one: a failing run never deletes the previous ones. Any failure makes the systemd unit fail.

Test a restore every month

libris-restore restores a backup into a separate, throwaway Compose project and checks that it works. It never touches your installation, and it never starts the worker, which would resume the saved jobs and call your model providers.

cd /opt/libris
project="$(sed -n 's/^LIBRIS_PROJECT=//p' .env | head -n 1)"
export LIBRIS_PRODUCTION_PROJECT="${project:-libris}"
export LIBRIS_RESTORE_COMPOSE_FILE="$PWD/docker-compose.yml"
export LIBRIS_RESTORE_ENV_FILE="$PWD/.env"
export LIBRIS_RESTORE_IMAGE="$(sed -n 's/^LIBRIS_IMAGE=//p' .env | head -n 1)"
backup/libris-restore --dry-run /mnt/libris-backup/libris/libris-<timestamp>
backup/libris-restore /mnt/libris-backup/libris/libris-<timestamp>

• --dry-run checks the checksums and the archive, and prints each Docker command without running it.
• The real run creates the project libris-restore-test with its own volumes, publishes the API on a random loopback port, restores the database and the books, runs the migrations and the API, then checks alembic check and /health and prints the number of users, books, passages, book files and source files.
• The project and its volumes are removed at the end. With --keep they stay for inspection: the script prints the Compose command of the test project (with its --file and --env-file); add port api 8088 to it to find the port, sign in, then run it with down --volumes to remove the project.

Compare the counts with your installation and write down the date of the test.

For a registry installation, the script lives in backup/. The variables above select the real Compose project (libris by default), configuration and pinned image. For an installation managed by the production deployment script, its defaults still match the deployment script.

Restore an installation

Do this after losing data, or to return to the state before a failed update. Use the same .env (at least the same SECRET_KEY) and an application version at least as recent as the one that made the backup.

1. Pause running books in the interface if Libris still runs.

2. Check the backup:

   backup=/mnt/libris-backup/libris/libris-<timestamp>
   (cd "$backup" && sha256sum --check SHA256SUMS)
   

3. Confirm that losing database changes since this backup is acceptable. On an installation managed by the production receiver, use the current operator CLI:

   librisctl restore "$backup/database.dump" --confirm --no-start
   

   It validates the archive before stopping services, takes a private rescue dump, replaces the application database and leaves services stopped. The rescue dump is not automatically pruned. Before a rollback, keep --no-start: otherwise the current image reapplies its migrations.

   Without that CLI, use a fresh database, not pg_restore --clean over a newer schema. --clean only drops objects listed in the old archive; newly added tables and foreign keys (for example review_links) can block it or remain behind. From the exact Libris directory, verify the target Compose project and database, then run each step only if the previous succeeds:

   docker compose exec -T database pg_restore --list < "$backup/database.dump" > /dev/null
   docker compose stop api worker
   mkdir -p backups && chmod 700 backups
   rescue=$(mktemp backups/pre-restore-XXXXXXXX.dump)
   chmod 600 "$rescue"
   docker compose exec -T database sh -c 'pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc' > "$rescue"
   test -s "$rescue"
   docker compose exec -T database sh -c 'case "$POSTGRES_DB" in ""|postgres|template0|template1) exit 64;; esac; dropdb -U "$POSTGRES_USER" -- "$POSTGRES_DB" && createdb -U "$POSTGRES_USER" --owner "$POSTGRES_USER" --template template0 -- "$POSTGRES_DB"'
   docker compose exec -T database sh -c 'pg_restore -U "$POSTGRES_USER" -d "$POSTGRES_DB" --no-owner --exit-on-error' < "$backup/database.dump"
   

   This replaces the dedicated application database, including objects not present in the dump. Other connections must be closed; the command does not forcibly disconnect them. Its recreated database uses the cluster’s default locale/encoding and the installation’s PostgreSQL owner. If the database has custom locale, extensions or other applications’ objects, have its operator adapt the procedure first. A failed restore leaves services stopped; keep both dumps.

4. Restore the books too if they were lost or damaged. This replaces the whole content of the volume:

   docker compose run --rm --no-deps -T api sh -c 'find /data -mindepth 1 -delete && tar -C /data -xzf -' < "$backup/books.tar.gz"
   

5. Start Libris again and check it:

   docker compose up -d --no-build --wait
   docker compose exec -T api alembic check
   

   Then sign in, open a book preview, test a provider connection and export an EPUB.

If .env was lost too, restore it first from config.env if your backups include it. Otherwise create a new one with python3 scripts/setup.py (the database password must then be the one of the restored volume) and enter every provider key again.

To rebuild a registry installation backed up with Libris 0.24.0 or later on a new machine, put the saved .env in place before anything starts. This sequence requires the image that made the backup to carry the --no-start installer (introduced in 0.24.0). For an older backup, first restore with the procedure for its release; do not run this sequence with its older installer. The saved LIBRIS_IMAGE must be a full sha256 digest of the Libris registry image. The installer reads that image, preserves .env and extracts the backup tools from it. Run:

Open a root shell and run the whole sequence there; /opt/libris and the backup archives are restricted to root:

sudo -i
mkdir -p -m 750 /opt/libris
install -m 0600 /path/to/config.env /opt/libris/.env
curl --fail --location --silent --show-error https://libris-translate.com/install.sh | bash -s -- --no-start
cd /opt/libris
docker compose up -d --wait database
backup=/mnt/libris-backup/libris/libris-YYYYMMDDTHHMMSSZ
(cd "$backup" && sha256sum --check SHA256SUMS)
docker compose exec -T database sh -c 'pg_restore -U "$POSTGRES_USER" -d "$POSTGRES_DB" --no-owner --exit-on-error' < "$backup/database.dump"
docker compose run --rm --no-deps -T api tar -C /data -xzf - < "$backup/books.tar.gz"
docker compose --profile codex up -d --no-build --wait
exit

--no-start (or LIBRIS_NO_START=1) pulls images without starting api, worker or migrate; only the explicit command above starts the database before restoration. The saved .env must contain the pinned images and original secrets. For another installation directory, set LIBRIS_HOME for the installer and run Compose there. Replace YYYYMMDDTHHMMSSZ with the UTC timestamp of the backup directory.

Never replace a database whose current data you want to keep without a verified rescue backup.

Roll back a failed update

Database migrations only go forward. To return to the previous version after an update that changed the database schema, you need the database as it was before the update.

Customer installation made with the registry bootstrap:

1. Stop the application: docker compose stop api worker.

2. Restore the database backup taken before the update (step 3 above). The books volume normally does not need restoring.

3. Start the previous version from the registry (the command keeps /opt/libris/.env and the restored volumes):

   curl --fail --location --silent --show-error https://libris-translate.com/install.sh \
     | sudo env LIBRIS_TAG=<previous version> bash
   

   The bootstrap extracts the matching Compose file from that exact image. Do not run it without LIBRIS_TAG until you deliberately want to return to the current stable release.

4. Check /health, sign in and resume the books.

Installation managed by the production deployment script: each deployment dumps the database just before migrating, into /opt/libris-production/backups/pre-<timestamp>-<commit>.<random>.dump (the last five are kept), and keeps the images it replaced.

• If the failed version did not change the schema, run libris-production-deploy --rollback (or librisctl rollback --confirm): it restarts the previous images. It refuses when the schema differs, without changing anything.
• Otherwise, restore the dump taken just before the faulty deployment (librisctl restore <dump> --confirm --no-start, or step 3 above with the Compose options shown in operations), skip the books, then run libris-production-deploy --rollback.

The pre-deployment dumps stay on the same disk as the installation: they protect an update, not the machine. Keep the nightly backup as well.