lua-resty-dymetrics

The use must have the root permission.

Prerequisites

  • The use must have the root permission.

  • The machine needs to be able to access the following domains to download packages and GPG keys:

    You can check the accessibility of the domains by running the following commands:

    curl https://openresty.org/package/pubkey.gpg
    curl https://pkg2.openresty.com/xray-priv-libs/
    

Recompile OpenResty

The Dynamic Metrics module requires a patch to the Nginx core, so OpenResty needs to be recompiled. This patch will be delivered via email.

The following example patches the Nginx core for OpenResty 1.19.9.1. For other versions, replace the source package version, the bundled Nginx version, and the corresponding patch file delivered by email. When installing the module, select a dymetrics-nginx-module package with the same Nginx version suffix. After patching, users can follow the original packaging process.

set -e
pkg=openresty-1.19.9.1
ngx=nginx-1.19.9

wget https://openresty.org/download/$pkg.tar.gz
tar -xf $pkg.tar.gz

cd $pkg/bundle/$ngx/
patch -p1 < ../../../nginx-1.19.9-proc_exit_handler.patch
cd ../../..

tar -czf $pkg.tar.gz  $pkg

Configure the software repository

First, configure the binary package repository using the following commands. Replace CLIENT_TOKEN with the valid token from your subscription email and CUSTOMER_NAME with the customer name specified in that email.

curl -o get-xray-priv-lib-repo.sh https://pkg2.openresty.com/scripts/get-xray-priv-lib-repo.sh

sudo bash get-xray-priv-lib-repo.sh -l lua-resty-dymetrics-CUSTOMER_NAME -t CLIENT_TOKEN

sudo bash get-xray-priv-lib-repo.sh -l dymetrics-nginx-module -t CLIENT_TOKEN

sudo bash get-xray-priv-lib-repo.sh -l openresty-priv-comm -t CLIENT_TOKEN

Installing packages

Dynamic metrics requires the following four components to be installed

  1. openresty-yajl
  2. openresty-yajl-devel
  3. dymetrics-nginx-module
  4. lua-resty-dymetrics-CUSTOMER_NAME

The lua-resty-dymetrics-CUSTOMER_NAME component is customized for each customer. Replace CUSTOMER_NAME in both the repository configuration and installation commands with the customer name specified in your subscription email.

OpenResty 1.19.9.x

The following installation commands correspond to the OpenResty 1.19.9.1 build example above.

For operating systems using the yum package manager, run the following command to install the private library.

sudo yum install -y dymetrics-nginx-module-1.19.9 openresty-yajl openresty-yajl-devel
sudo yum install -y lua-resty-dymetrics-CUSTOMER_NAME

For operating systems using the dnf package manager, execute the following command to install the private libraries.

sudo dnf install -y dymetrics-nginx-module-1.19.9 openresty-yajl openresty-yajl-devel
sudo dnf install -y lua-resty-dymetrics-CUSTOMER_NAME

For operating systems using the apt package manager, execute the following command to install the private libraries.

sudo apt-get install -y dymetrics-nginx-module-1.19.9 openresty-yajl openresty-yajl-dev
sudo apt-get install -y lua-resty-dymetrics-CUSTOMER_NAME

OpenResty 1.31.8.x

For systems using the yum package manager, run the following commands to install the private library:

sudo yum install -y dymetrics-nginx-module-1.31.8 openresty-yajl openresty-yajl-devel
sudo yum install -y lua-resty-dymetrics-CUSTOMER_NAME

For systems using the dnf package manager, run the following commands to install the private library:

sudo dnf install -y dymetrics-nginx-module-1.31.8 openresty-yajl openresty-yajl-devel
sudo dnf install -y lua-resty-dymetrics-CUSTOMER_NAME

For systems using the apt package manager, run the following commands to install the private library:

sudo apt-get install -y dymetrics-nginx-module-1.31.8 openresty-yajl openresty-yajl-dev
sudo apt-get install -y lua-resty-dymetrics-CUSTOMER_NAME

Usage

You need to add the following configuration items to the configuration file nginx.conf before using it.

  1. load_module directive to load the dynamic module ngx_http_lua_dymetrics_module.
  2. The lua_package_path and lua_package_cpath directives specify the path to search for lua files and lua-loaded binary shared libraries.
  3. lua_shared_dymetrics directive defines a shared memory area to be used by dynamic metrics.
load_module /usr/local/openresty/nginx/modules/ngx_http_lua_dymetrics_module.so;

http {
    lua_package_path   "/usr/local/openresty/site/lualib/?.ljbc;;";
    lua_package_cpath  "/usr/local/openresty-yajl/lib/?.so;/usr/local/openresty/site/lualib/?.so;;";
    lua_shared_dymetrics  dymetrics 120M;

    server {
        listen       80;
        server_name  localhost;
        set $app_uid "test.openresty.com";

        log_by_lua_block {
            if ngx.ctx.no_dymetrics_log then
                return
            end

            local ts = ngx.time()
            ts = ts - (ts % 60)

            local request_log = require "resty.dymetrics_http_reqs".request_log
            local conn_log= require "resty.dymetrics_http_conns".request_log
            local lua_vars = {}
            local app_id = 0

            request_log(ts, app_id, lua_vars)
            conn_log(ts, app_id, lua_vars)
        }

        location /metrics {
            allow 127.0.0.1;
            allow 18.140.243.108; # just an example
            deny all;

            content_by_lua_block {
                ngx.ctx.no_dymetrics_log = true
                local report_req_data = require "resty.dymetrics_http_reqs".report_data_prometheus
                local report_conn_data = require "resty.dymetrics_http_conns".report_data_prometheus
                local ts = ngx.time()
                ts = ts - 60 - (ts % 60)

                local data, err = report_req_data(ts, 0, "")
                if err ~= nil then
                   ngx.log(ngx.ERR, "prometheus: failed to get http request data:", err)
                else
                    ngx.say(data)
                end

                local data, err = report_conn_data(ts, 0, "")
                if err ~= nil then
                   ngx.log(ngx.ERR, "prometheus: failed to get http connection data:", err)
                else
                    ngx.say(data)
                end
            }
        }

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

Lua APIs

dymetrics_http_reqs module

request_log

syntax: ok, err = reqs.request_log(time_key, app_id, lua_variables)

context: log_by_lua*

Calling this interface will add the current request to the statistics. If there is an error, it will return nil and the corresponding error message.

The value of time_key is the starting value of the statistics cycle. For example, if 60s is a statistics cycle, then “time_key” can be obtained as follows: time_key = ngx.time() - ngx.time() % 60. app_id is a key for Prometheus output. If app_id is not distinguished, pass value 0. lua_variables is an extension variable that is currently not used and requires passing an empty Lua table.

Here is an example:

log_by_lua_block {
    local ts = ngx.time()
    ts = ts - (ts % 60)

    local request_log = require "resty.dymetrics_http_reqs".request_log
    local lua_vars = {}
    local app_id = 0

    request_log(ts, app_id, lua_vars)
}

report_data

syntax: ok, err = reqs. report_data(time_key, node_id, partition_id, gateway_id)

context: content_by_lua_block*,ngx.timer.*

Get the dynamic indicators for the statistics cycle that starts with time_key. This time_key is for the previous statistics cycle, not the current one, because the current statistics cycle has not yet ended, and new requests will still be added to it. node_id represents the current machine’s ID. If it does not exist, it should be passed as 0. partition_id is a string representing the current partition’s ID. If it does not exist, it can be passed as nil. gateway_id is the ID of the gateway that the current node belongs to. If it does not exist, it can be passed as nil.

Here is an example:

content_by_lua_block {
    local report_req_data = require "resty.dymetrics_http_reqs".report_data
    local ts = ngx.time()
    ts = ts - 60 - (ts % 60)

    local data, err = report_req_data(ts, 0, "")
    if err ~= nil then
       ngx.log(ngx.ERR, "prometheus: failed to get http request data:", err)
    else
        ngx.say(data)
    end
}

report_data_prometheus

syntax: ok, err = reqs.report_data_prometheus(time_key, node_id, partition_id, gateway_id)

context: content_by_lua_block*,ngx.timer.*

Get the dynamic indicators for the statistics cycle that starts with time_key. This time_key is for the previous statistics cycle, not the current one, because the current statistics cycle has not yet ended, and new requests will still be added to it. node_id represents the current machine’s ID. If it does not exist, it should be passed as 0. partition_id is a string representing the current partition’s ID. If it does not exist, it can be passed as nil. gateway_id is the ID of the gateway that the current node belongs to. If it does not exist, it can be passed as nil.

Here is an example:

content_by_lua_block {
    local report_req_data = require "resty.dymetrics_http_reqs".report_data_prometheus
    local ts = ngx.time()
    ts = ts - 60 - (ts % 60)

    local data, err = report_req_data(ts, 0, "")
    if err ~= nil then
       ngx.log(ngx.ERR, "prometheus: failed to get http request data:", err)
    else
        ngx.say(data)
    end
}

dymetrics_http_conns module

request_log

syntax: ok, err = conns.request_log(time_key, app_id, lua_variables)

context: log_by_lua*

Calling this interface will add the current request to the statistics. If there is an error, it will return nil and the corresponding error message.

The value of time_key is the starting value of the statistics cycle. For example, if 60s is a statistics cycle, then “time_key” can be obtained as follows: time_key = ngx.time() - ngx.time() % 60. app_id is a key for Prometheus output. If app_id is not distinguished, pass value 0. lua_variables is an extension variable that is currently not used and requires passing an empty Lua table.

Here is an example:

log_by_lua_block {
    local ts = ngx.time()
    ts = ts - (ts % 60)

    local conn_log = require "resty.dymetrics_http_conns".request_log
    local lua_vars = {}
    local app_id = 0

    request_log(ts, app_id, lua_vars)
}

report_data

syntax: ok, err = conns.report_data(time_key, node_id, partition_id, gateway_id)

context: content_by_lua_block*,ngx.timer.*

Get the dynamic indicators for the statistics cycle that starts with time_key. This time_key is for the previous statistics cycle, not the current one, because the current statistics cycle has not yet ended, and new requests will still be added to it. node_id represents the current machine’s ID. If it does not exist, it should be passed as 0. partition_id is a string representing the current partition’s ID. If it does not exist, it can be passed as nil. gateway_id is the ID of the gateway that the current node belongs to. If it does not exist, it can be passed as nil.

Here is an example:

content_by_lua_block {
    local report_req_data = require "resty.dymetrics_http_conns".report_data
    local ts = ngx.time()
    ts = ts - 60 - (ts % 60)

    local data, err = report_req_data(ts, 0, "")
    if err ~= nil then
       ngx.log(ngx.ERR, "prometheus: failed to get http request data:", err)
    else
        ngx.say(data)
    end
}

report_data_prometheus

syntax: ok, err = conns.report_data_prometheus(time_key, node_id, partition_id, gateway_id)

context: content_by_lua_block*,ngx.timer.*

Get the dynamic indicators for the statistics cycle that starts with time_key. This time_key is for the previous statistics cycle, not the current one, because the current statistics cycle has not yet ended, and new requests will still be added to it. node_id represents the current machine’s ID. If it does not exist, it should be passed as 0. partition_id is a string representing the current partition’s ID. If it does not exist, it can be passed as nil. gateway_id is the ID of the gateway that the current node belongs to. If it does not exist, it can be passed as nil.

Here is an example:

content_by_lua_block {
    local report_req_data = require "resty.dymetrics_http_conns".report_data_prometheus
    local ts = ngx.time()
    ts = ts - 60 - (ts % 60)

    local data, err = report_req_data(ts, 0, "")
    if err ~= nil then
       ngx.log(ngx.ERR, "prometheus: failed to get http request data:", err)
    else
        ngx.say(data)
    end
}