MongoDb Postgres Backup Restore & Retention

Last updated: 2024-05-29 14:46

To make sure the data is persistent on both healthdata.be's and data provider's side the back-up plan was confirmed and developed and tested as well, the postgres and the MongoDb will be taken the whole back-up on the daily basis, this will be taken care of the timer-service that has been recently introduced.

Last 30 days back-up will be kept in the same server share or in the local path, to enable the backup we have introduced the new ansible role called “mongo-postgres-backup”, and to invoke the backup option the following variables needs to be defined on the “Host_var” file on hd-inventory to the specific deployment.

Enabling the backup part on the target server: As soon as the ansible role added to the respective playbook and the above backup type added to the hd-inventory host vars to the main/master branch, this can be enabled automatically on the hd-updater daily run or manually triggering the hd-updater on the target side server.

  • When DP do not have the external share,

then it’s better to keep the backup of the postgres-mongodb dumps, on different directory on the same server. To enable this backup type, we must add the below variable on to the respective host-vars in the hd-inventory.

backup_mount_type: "VmBackup"

----Default backup path is "/hd4dp-backup"

  • When DP do not have both external share and Vm backup,

then it’s better to keep backup of the postgres-mongodb dumps, on different directory on the same server. To enable this backup type, we have to add the below variable on to the respective host-vars in the hd-inventory.

backup_mount_type : "local"

local_mount_path : /tmpBackup

  • When DP have External share (samba),

then the below variable needs to be added to the respective hd-inventory host_var file.

backup_mount_type: "samba"

backup_mount_type: “samba”

samba_bkup_username: “user_name”

samba_bkup_domain: “example.com”

  • When DP provides External share (nfs),

then the below variable needs to be added to the respective hd-inventory host_var file.

backup_mount_type: "nfs"

nfs_bkup_src: "0.0.0.0:/nfs/mount-share"

----Default backup path is "/hd4dp-backup"

Restore:

AS of the agreed approach is to restore the backup-dump manually by using the retention-script (restore_mongodb.sh & restore_postgresdb.sh) it can be found on the target server “/opt/hd-all/hd-backup”, by default it will restore the latest backup but if we wish to restore the particular date back this can be given as a input parameter refer below.

Restore Script Path on the target server:

/opt/hd-all/hd-backup

Mongo Restore Input param:

"Provide bkp file with full path name for restore e.g. ${local_mount_path}/hd4dp-backup/mongodbBackup/mongo-20220510_2004.tar.gz" & ) to the script.

Mongo restore script:

restore_mongodb.sh

Postrges restore input Param:

"Provide bkp file with fullpath name for pg_wal restore e.g. ${local_mount_path}/hd4dp-backup/postgresBackup/pg_wal_latest.tar.gz"

“Provide bkp file with fullpath name for pg_wal restore e.g. {local_mount_path}/hd4dp-backup/postgresBackup/base_latest.tar.gz

Postgres restore script:

restore_postgresdb.sh

Note:

The postgres backup service timer will be scheduled by 15:00 aprox, the Mongodb backup service timer will be scheduled by 22:00, backup retention will perform few basic checks ex it will check if the space is sufficient to store the current day backup.