OpenResty Edge Configuration Mirror Backed by YAML Files

We provides a command line tool for updating configurations in OpenResty Edge using local YAML-backed config mirroring. Configurations are divided by partitions, with each set of configurations corresponding to a partition in OpenResty Edge. Local configurations can be version-controlled using programs like Git.

The following content in the document explains how to use this tool.

Create an API Token

  1. First, log in to the OpenResty Edge Admin Console.
  2. Then click on the username in the upper right corner: USERNAME > API Tokens > New API Token > Copy API Token.

Update or add configuration to OpenResty Edge

We will separate the command line tools for handling configuration and the initial configuration into different packages, to prevent the initial configuration in the newly released packages from overwriting your updated configurations.

  • Download the openresty-edge-cmd-tools-VERSION.tar.gz and openresty-edge-cmd-configs-VERSION.tar.gz from OpenResty Download Center.The openresty-edge-cmd-tools-VERSION.tar.gz contains command line tools. The openresty-edge-cmd-configs-VERSION.tar.gz consists of configurations. If you are migrating from another platform to OpenResty Edge, we can perform a conversion with your configurations and provide you with examples.

  • Upload the openresty-edge-cmd-tools-VERSION.tar.gz and openresty-edge-cmd-configs-VERSION.tar.gz package to any machine that has Python3 installed and can access Edge Admin.

  • You can extract the contents of openresty-edge-cmd-configs-VERSION.tar.gz into the directory where openresty-edge-cmd-tools-VERSION.tar.gz has been extracted, forming the following directory structure. Alternatively, you can extract the two packages into separate directories and then specify the configuration path through the -i option of the command line tool.

  • Extracting openresty-edge-cmd-configs-VERSION.tar.gz yields the following directories:

    - openresty-edge-cmd-configs-VERSION/
    |-- configs/
      |-- global_lua_modules/
        |-- util.lua
        |-- uuid.lua
        |-- ...
      |-- upstreams/
        |-- upstream-a.yaml
        |-- upstream-b.yaml
        |-- ...
      |-- page_rules.yaml
      |-- global_configs.yaml
    

    The configs/ directory contains all the configurations for a partition. The global_lua_modules/ directory holds the global Lua modules, with each Lua file representing a Lua module. The upstreams/ directory contains the upstreams, and you can split the upstreams into multiple files for easier management. The page_rules.yaml file contains page rules, which will be added to the partition-related HTTP applications. The global_configs.yaml file includes global configurations such as global user variables.

  • Extracting openresty-edge-cmd-tools-VERSION.tar.gz yields the following directories:

    - openresty-edge-cmd-tools-VERSION/
    |-- edge-config.py
    |-- gen-cert.sh
    

    The edge-config.py is a command line tool you can execute to manage partition configurations. You can specify the configuration path via the -i option of the command line tool; when the -i option is not specified, the default is to use the configs/ directory located in the same directory as edge-config.py.

    You may refer to the configuration files and directory structure we have provided to generate configurations for other environments. Should you encounter any issues, or if the command line tool requires support for additional configurations, please do not hesitate to contact us. We will be more than happy to provide you with the necessary support.

  • Install the dependencies required for the command line tool

    pip3 install openresty-edge-sdk ruamel.yaml
    

    The version of openresty-edge-sdk should be greater than or equal to 1.2.51.

  • Run the command line tool

    If you want to see the usage of this command line tool, you can execute the following command:

    python3 edge-config.py -h
    

    The output of this command looks like this:

    usage: edge-config.py [-h] -t API_TOKEN -u ADMIN_URL [-s] [-c] [-i CONFIGS_PATH] [-l LOCATION]
                          [-d DOMAIN] (-p PARTITION_ID | -n PARTITION_NAME)
    
    Update or add OpenResty Edge configuration.
    
    optional arguments:
      -h, --help            show this help message and exit
      -t API_TOKEN, --api-token API_TOKEN
                            specify the API Token for sending the request
      -u ADMIN_URL, --admin-url ADMIN_URL
                            specify the URL of the OpenResty Edge Admin. For example,
                            https://admin.com:443
      -s, --disable-ssl-verify
                            turn off SSL verify for requests to access OpenResty Edge Admin
      -c, --cleanup         based on the location. This option allows for the cleanup of page rules,
                            application upstreams, global user variables, and resetting the access log
                            format for partitions. It can also be used independently of the location
      -i CONFIGS_PATH, --configs-path CONFIGS_PATH
                            specify the path to the configuration file; if not specified, the default is
                            the 'configs' directory in the same directory as the current script
      -l LOCATION, --location LOCATION
                            specify the configuration name that needs to be updated, supported:
                            page_rules, upstreams, global_lua_modules, global_configs
      -d DOMAIN, --domain DOMAIN
                            specify a domain name. When an HTTP application containing this domain exists,
                            it will be updated; otherwise, a new application will be created
      -p PARTITION_ID, --partition-id PARTITION_ID
                            specify the id of the partition where you want to add or update the
                            configuration
      -n PARTITION_NAME, --partition-name PARTITION_NAME
                            specify the name of the partition where you want to add or update the
                            configuration
    

    The configuration can be updated to a specified partition by specifying the -p PARTITION_ID or -n PARTITION_NAME option.

    Example:

    python3 edge-config.py -t 3ed00172-280e-4e43-a4ef-3c2fae5669ca -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com
    

    This command will update the configuration to the default partition (named default, with an id of 1), and if the test.com application does not exist, it will create it.

If you continue to encounter the error No module named 'MODULE_NAME', please use the following command to install the dependencies:

pip3 install MODULE_NAME

Please change the MODULE_NAME in the command to the module name.

More examples:

# remove: lua modules, global user variables, application
# reset: access log formats
python3 edge-config.py -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com -c

# remove all page rules from app "test.com"
python3 edge-config.py -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l page_rules -d test.com -c

# remove all Lua modules from partition 1(default)
python3 edge-config.py -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l global_lua_modules -c

# update/insert all configs to partition 1(default)
python3 edge-config.py -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com

# update/insert page rules to app "test.com"
python3 edge-config.py -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l page_rules -d test.com

Configuration Example

Global Configuration

Access log formats: all access log formats make up an array.

access_log_formats:
- default: false
  escape: json
  name: example
  format: |-
    {
        "time_local": "$time_local",
        "method": "$request_method",
        "request": "$request",
        "transid": "$or_global_user_variable_uuid",
        "upstream_header_time": "$upstream_header_time"
    }

The default field controls whether to set the method log format to default. If set to default, applications that do not specify an access log format will use this format. The content of the access log format is specified through the format field, and a global user variable named uuid is used in the log, represented by $or_global_user_variable_uuid.

Custom shared zone, all shared zones make up an array:

custom_shared_zone:
- size_unit: m
  name: cache_shared_dict
  size: 1
- size_unit: m
  name: rec_cache_shared_dict
  size: 1

Global custom variables, all variables make up an array:

user_variables:
- name: uuid
  type: string
  default: '-'

This example defines a global user variable named uuid. When using it in access logs or Edgelang, it should be replaced with $or_global_user_variable_uuid.

Global Lua Modules

Here is a Lua module named hello.lua:

local _M = {}

function _M.go()
    ngx.say("hello")
end

return _M

Lua modules are located in the global_lua_modules directory, and the command line tool will automatically update all the files in this directory to OpenResty Edge.

Page Rules

- enable_rule: true
  actions:
    set-access-log-off: {}
    print:
      content_type: text/html
      msg: favicon.ico and robots.txt
    exit:
      code: 200
  conditions:
  - op: eq
    var: uri
    vals:
      - /favicon.ico
      - /robots.txt
  order: 1
  comment: ''
- enable_rule: true
  actions:
    "user-code":
      el: |-
        {
        my Str $target_upstream;
        my Str $transid;

        true =>
            foreign-call(module: "example", func: "go"),
            $target_upstream = ctx-var("lm_target_upstream"),
            $transid = ctx-var("lm_transid"),
            $or_global_user_variable_uuid = $transid,
            set-proxy-timeouts(connect: 12 [s], send: 60 [s], read: 60 [s]),
            set-upstream-name($target_upstream),
            done;
        }
  comment: ''

Page rules are ordered, and the sequence in the configuration file will be maintained when updated to the Edge Admin. This example includes two page rules. The first rule determines whether the URI is /favicon.ico or /robots.txt, and if so, it disables access logging and inputs specified content. The second rule unconditionally executes a segment of Edgelang code, which sets the global user variable $or_global_user_variable_uuid and specifies the upstream’s name to be accessed.

All page rules make up an array, with each page rule being an element of the array.

Upstreams

app_upstream_name1:
  ssl: false
  disable_ssl_verify: false
  servers:
  - host: 1.1.1.1
    port: 80
  - host: 1.1.1.2
    port: 80
app_upstream_name2:
  ssl: false
  disable_ssl_verify: false
  servers:
  - host: 1.1.1.3
    port: 80

This upstream configuration example includes two upstreams, app_upstream_name1 and app_upstream_name2. One contains two upstream servers, while the other has one upstream server.

In the upstream file, they are organized in the form of a hash table, with the key being the name of the upstream and the value being the other details of the upstream.

Error Message Example

For configuration file errors, the command line tool will attempt to provide prompts indicating the specific line number where an error occurred.

[!] Error: invalid port in upstream upstream_name_foo: 70000, file: foo.yaml, line: 6.

There are also some other errors, such as the error below indicating that the input configuration path is incorrect; the command line tool cannot find this path.

[!] Error: bad configs path: /bad/path.