Migrating OpenResty Edge Log Server Database Data

When the database space used by OpenResty Edge Log Server is insufficient or data needs to be migrated to other devices, you can refer to this document for operation. We provide three migration methods. Please choose the most suitable solution based on your actual situation.

Note: The following operations involve deletion. Please operate with caution to avoid accidentally deleting important data.

Method 1: Migrating the Complete Data Directory

Applicable Scenarios

  • Insufficient hard disk space, wanting to migrate data to a new hard disk on the same machine.
  • Or insufficient hard disk space, wanting to migrate to a hard disk on another machine.
  • Or no disk space issues, simply wanting to perform data migration.
  • Suitable for local migration or cross-machine migration.

Steps for Migration Between Different Hard Disks on the Same Machine

  1. If the Log Server database service is still working normally, stop the Log Server database service to avoid data inconsistency:

    # For PostgreSQL 15
    sudo systemctl stop openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl stop openresty-postgresql12
    
  2. Copy the data to a hard disk with sufficient space:

    # For PostgreSQL 15
    sudo cp -p -R /var/postgres15/data/ /path/to/new/data
    
    # For PostgreSQL 12
    sudo cp -p -R /var/postgres12/data/ /path/to/new/data
    
  3. Rename the original data directory:

    # For PostgreSQL 15
    sudo mv /var/postgres15/data/ /var/postgres15/data-bak
    
    # For PostgreSQL 12
    sudo mv /var/postgres12/data/ /var/postgres12/data-bak
    
  4. Create a symbolic link: (If space is severely insufficient, you can delete some logs or other files to free up space)

    # For PostgreSQL 15
    sudo ln -s /path/to/new/data /var/postgres15/data
    
    # For PostgreSQL 12
    sudo ln -s /path/to/new/data /var/postgres12/data
    
  5. After confirming that the new data is correct, you can delete the old data:

    # For PostgreSQL 15
    sudo rm -rf /var/postgres15/data-bak
    
    # For PostgreSQL 12
    sudo rm -rf /var/postgres12/data-bak
    
  6. (Optional) Restart the Log Server database service:

    # For PostgreSQL 15
    sudo systemctl start openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl start openresty-postgresql12
    

Steps for Cross-Machine Migration

  1. Stop the Log Server database service on the source machine to avoid data inconsistency:

    # For PostgreSQL 15
    sudo systemctl stop openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl stop openresty-postgresql12
    
  2. Install the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/

  3. Stop the Log Server database service on the target machine:

    # For PostgreSQL 15
    sudo systemctl stop openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl stop openresty-postgresql12
    
  4. Delete or move the initial data on the target machine: (Please triple-check the machine you are operating on to avoid accidentally deleting valid data)

    Delete:

    # For PostgreSQL 15
    sudo rm -rf /var/postgres15/data
    
    # For PostgreSQL 12
    sudo rm -rf /var/postgres12/data
    

    Move:

    # For PostgreSQL 15
    sudo mv /var/postgres15/data /var/postgres15/data-init
    
    # For PostgreSQL 12
    sudo mv /var/postgres12/data /var/postgres12/data-init
    
  5. Use a secure file transfer method (such as scp or rsync) to copy data from the source machine to the target machine:

    scp:

    # For PostgreSQL 15
    scp -r /var/postgres15/data user@target_host:/var/postgres15/data
    
    # For PostgreSQL 12
    scp -r /var/postgres12/data user@target_host:/var/postgres12/data
    

    rsync:

    # For PostgreSQL 15
    rsync -avz --owner --group --perms /var/postgres15/data user@target_host:/var/postgres15/data
    
    # For PostgreSQL 12
    rsync -avz --owner --group --perms /var/postgres12/data user@target_host:/var/postgres12/data
    
  6. Restart the database service on the target machine:

    # For PostgreSQL 15
    sudo systemctl restart openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl restart openresty-postgresql12
    
  7. Log in to the machine where the Log Server is located and modify the configuration to use the new database service:

    sudo vi /usr/local/oredge-log-server/conf/config.ini
    

Method 2: Migration Using Master-Slave Replication

Applicable Scenarios

  • The Log Server database service is currently running normally.
  • Suitable for cross-machine migration.

Operation Steps

  1. Install the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/

  2. Configure the source machine as the master and the target machine as the slave according to the document Configuring OpenResty Edge Database High Availability with Interactive Scripts or OpenResty Edge Database High Availability.

  3. Stop the master on the source machine:

    # For PostgreSQL 15
    sudo systemctl stop openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl stop openresty-postgresql12
    
  4. Promote the slave to master on the target machine:

    # For PostgreSQL 15
    sudo -u postgres /usr/local/openresty-postgresql15/bin/pg_ctl promote -D /var/postgres15/data
    
    # For PostgreSQL 12
    sudo -u postgres /usr/local/openresty-postgresql12/bin/pg_ctl promote -D /var/postgres12/data
    
  5. Log in to the machine where the Log Server is located and modify the configuration to use the new database service:

    sudo vi /usr/local/oredge-log-server/conf/config.ini
    

Method 3: Migration by Exporting and Importing Data

Applicable Scenarios

  • The Log Server database service is currently running normally.
  • There is enough space on the source machine to save the exported data.
  • Suitable for cross-machine migration.

Operation Steps

  1. Stop the Log Server service to prevent new data from being written to the database:

    sudo systemctl stop oredge-log-server
    
  2. Export data from the source machine:

    # For PostgreSQL 15
    /usr/local/openresty-postgresql15/bin/pg_dump or_edge_log_server -C -Upostgres | gzip > or_edge_log_server-`date +%F`.gz
    
    # For PostgreSQL 12
    /usr/local/openresty-postgresql12/bin/pg_dump or_edge_log_server -C -Upostgres | gzip > or_edge_log_server-`date +%F`.gz
    
  3. Install the Log Server database service on the target machine; refer to the document: https://doc.openresty.com/en/edge/edge-installation-upgrade/edge-installation/

  4. Stop the Log Server database service on the target machine:

    # For PostgreSQL 15
    sudo systemctl stop openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl stop openresty-postgresql12
    
  5. Delete the initial data on the target machine: (Please triple-check the machine you are operating on to avoid accidentally deleting valid data)

    # For PostgreSQL 15
    sudo rm -rf /var/postgres15/data
    
    # For PostgreSQL 12
    sudo rm -rf /var/postgres12/data
    
  6. Transfer the exported data to the target machine

  7. Import data on the target machine:

    # For PostgreSQL 15
    zcat or_edge_log_server-{date}.gz | /usr/local/openresty-postgresql15/bin/psql -Upostgres
    
    # For PostgreSQL 12
    zcat or_edge_log_server-{date}.gz | /usr/local/openresty-postgresql12/bin/psql -Upostgres
    
  8. Start the database service on the target machine:

    # For PostgreSQL 15
    sudo systemctl start openresty-postgresql15
    
    # For PostgreSQL 12
    sudo systemctl start openresty-postgresql12
    
  9. Modify the Log Server configuration to use the new database service:

    sudo vi /usr/local/oredge-log-server/conf/config.ini