Migrating OpenResty Edge Log Server Database Data
When the database space used by OpenResty Edge Log Server is insufficient or data needs to be migrated to other devices, you can refer to this document for operation. We provide three migration methods. Please choose the most suitable solution based on your actual situation.
Note: The following operations involve deletion. Please operate with caution to avoid accidentally deleting important data.
Method 1: Migrating the Complete Data Directory
Applicable Scenarios
- Insufficient hard disk space, wanting to migrate data to a new hard disk on the same machine.
- Or insufficient hard disk space, wanting to migrate to a hard disk on another machine.
- Or no disk space issues, simply wanting to perform data migration.
- Suitable for local migration or cross-machine migration.
Steps for Migration Between Different Hard Disks on the Same Machine
If the Log Server database service is still working normally, stop the Log Server database service to avoid data inconsistency:
# For PostgreSQL 15 sudo systemctl stop openresty-postgresql15 # For PostgreSQL 12 sudo systemctl stop openresty-postgresql12Copy the data to a hard disk with sufficient space:
# For PostgreSQL 15 sudo cp -p -R /var/postgres15/data/ /path/to/new/data # For PostgreSQL 12 sudo cp -p -R /var/postgres12/data/ /path/to/new/dataRename the original data directory:
# For PostgreSQL 15 sudo mv /var/postgres15/data/ /var/postgres15/data-bak # For PostgreSQL 12 sudo mv /var/postgres12/data/ /var/postgres12/data-bakCreate a symbolic link: (If space is severely insufficient, you can delete some logs or other files to free up space)
# For PostgreSQL 15 sudo ln -s /path/to/new/data /var/postgres15/data # For PostgreSQL 12 sudo ln -s /path/to/new/data /var/postgres12/dataAfter confirming that the new data is correct, you can delete the old data:
# For PostgreSQL 15 sudo rm -rf /var/postgres15/data-bak # For PostgreSQL 12 sudo rm -rf /var/postgres12/data-bak(Optional) Restart the Log Server database service:
# For PostgreSQL 15 sudo systemctl start openresty-postgresql15 # For PostgreSQL 12 sudo systemctl start openresty-postgresql12
Steps for Cross-Machine Migration
Stop the Log Server database service on the source machine to avoid data inconsistency:
# For PostgreSQL 15 sudo systemctl stop openresty-postgresql15 # For PostgreSQL 12 sudo systemctl stop openresty-postgresql12Install the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/
Stop the Log Server database service on the target machine:
# For PostgreSQL 15 sudo systemctl stop openresty-postgresql15 # For PostgreSQL 12 sudo systemctl stop openresty-postgresql12Delete or move the initial data on the target machine: (Please triple-check the machine you are operating on to avoid accidentally deleting valid data)
Delete:
# For PostgreSQL 15 sudo rm -rf /var/postgres15/data # For PostgreSQL 12 sudo rm -rf /var/postgres12/dataMove:
# For PostgreSQL 15 sudo mv /var/postgres15/data /var/postgres15/data-init # For PostgreSQL 12 sudo mv /var/postgres12/data /var/postgres12/data-initUse a secure file transfer method (such as scp or rsync) to copy data from the source machine to the target machine:
scp:
# For PostgreSQL 15 scp -r /var/postgres15/data user@target_host:/var/postgres15/data # For PostgreSQL 12 scp -r /var/postgres12/data user@target_host:/var/postgres12/datarsync:
# For PostgreSQL 15 rsync -avz --owner --group --perms /var/postgres15/data user@target_host:/var/postgres15/data # For PostgreSQL 12 rsync -avz --owner --group --perms /var/postgres12/data user@target_host:/var/postgres12/dataRestart the database service on the target machine:
# For PostgreSQL 15 sudo systemctl restart openresty-postgresql15 # For PostgreSQL 12 sudo systemctl restart openresty-postgresql12Log in to the machine where the Log Server is located and modify the configuration to use the new database service:
sudo vi /usr/local/oredge-log-server/conf/config.ini
Method 2: Migration Using Master-Slave Replication
Applicable Scenarios
- The Log Server database service is currently running normally.
- Suitable for cross-machine migration.
Operation Steps
Install the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/
Configure the source machine as the master and the target machine as the slave according to the document Configuring OpenResty Edge Database High Availability with Interactive Scripts or OpenResty Edge Database High Availability.
Stop the master on the source machine:
# For PostgreSQL 15 sudo systemctl stop openresty-postgresql15 # For PostgreSQL 12 sudo systemctl stop openresty-postgresql12Promote the slave to master on the target machine:
# For PostgreSQL 15 sudo -u postgres /usr/local/openresty-postgresql15/bin/pg_ctl promote -D /var/postgres15/data # For PostgreSQL 12 sudo -u postgres /usr/local/openresty-postgresql12/bin/pg_ctl promote -D /var/postgres12/dataLog in to the machine where the Log Server is located and modify the configuration to use the new database service:
sudo vi /usr/local/oredge-log-server/conf/config.ini
Method 3: Migration by Exporting and Importing Data
Applicable Scenarios
- The Log Server database service is currently running normally.
- There is enough space on the source machine to save the exported data.
- Suitable for cross-machine migration.
Operation Steps
Stop the Log Server service to prevent new data from being written to the database:
sudo systemctl stop oredge-log-serverExport data from the source machine:
# For PostgreSQL 15 /usr/local/openresty-postgresql15/bin/pg_dump or_edge_log_server -C -Upostgres | gzip > or_edge_log_server-`date +%F`.gz # For PostgreSQL 12 /usr/local/openresty-postgresql12/bin/pg_dump or_edge_log_server -C -Upostgres | gzip > or_edge_log_server-`date +%F`.gzInstall the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/
Stop the Log Server database service on the target machine:
# For PostgreSQL 15 sudo systemctl stop openresty-postgresql15 # For PostgreSQL 12 sudo systemctl stop openresty-postgresql12Delete the initial data on the target machine: (Please triple-check the machine you are operating on to avoid accidentally deleting valid data)
# For PostgreSQL 15 sudo rm -rf /var/postgres15/data # For PostgreSQL 12 sudo rm -rf /var/postgres12/dataTransfer the exported data to the target machine
Import data on the target machine:
# For PostgreSQL 15 zcat or_edge_log_server-{date}.gz | /usr/local/openresty-postgresql15/bin/psql -Upostgres # For PostgreSQL 12 zcat or_edge_log_server-{date}.gz | /usr/local/openresty-postgresql12/bin/psql -UpostgresStart the database service on the target machine:
# For PostgreSQL 15 sudo systemctl start openresty-postgresql15 # For PostgreSQL 12 sudo systemctl start openresty-postgresql12Modify the Log Server configuration to use the new database service:
sudo vi /usr/local/oredge-log-server/conf/config.ini