Edge: OpenResty Edge™ Log Collection (Host Deployment)

Mon, 01 Jan 0001 00:00:00 +0000

We recommend using Filebeat for Openresty Edge error log and access log collection.

Filebeat Installation

Install Filebeat software on all Openresty Edge Node machines. Please refer to Filebeat installation.

Filebeat Configuration

The following configuration is a simple example of collecting error and access logs on the Openresty Edge Node and sending them to Kafka:

filebeat.inputs:
- type: log
enabled: true
paths:
- /usr/local/oredge-node/logs/access.log*
- /usr/local/oredge-node/logs/error.log*
output.kafka:
enabled: true
# fill in the kafka broker address, this can also be multiple broker addresses
hosts: ["127.0.0.1:9092"]
topic: edge_node_logs
logging.level: error
# fill in the local host name or IP
name: 127.0.0.1

Save the above to /etc/filebeat/filebeat.yml and start the Filebeat service, which will send the logs to Kafka once successfully started.

sudo systemctl start filebeat

Validation

We can use the command line tool kafka-console-consumer.sh provided by Kafka to check if the logs were sent successfully.

On a machine with Kafka installed, execute the following command (replace the Kafka installation directory as appropriate), and a log output indicates that Filebeat is working properly.

$ /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic edge_node_logs --from-beginning --max-messages 1
{"@timestamp":"2022-12-27T07:06:53.045Z","@metadata":{"beat":"filebeat","type":"_doc","version":"8.5.3"},"message":"127.0.0.1 - - [14/Dec/2022:16:02:36 +0800] test.com \"GET http://test.com/ HTTP/1.1\" 200 22 0.000 \"-\" \"curl/7.61.1\" - - -","input":{"type":"log"},"ecs":{"version":"8.0.0"},"host":{"name":"127.0.0.1"},"agent":{"version":"8.5.3","ephemeral_id":"fd94a28e-91fe-4365-9584-fe6beb258751","id":"e84c2374-3272-430e-bfe6-84ad2b24b845","name":"127.0.0.1","type":"filebeat"},"log":{"offset":0,"file":{"path":"/usr/local/oredge-node/logs/access.log_20221214.160000"}}}

Edge: OpenResty Edge™ Log Collection (Kubernetes Deployment)

Mon, 01 Jan 0001 00:00:00 +0000

In a Kubernetes environment, each OpenResty Edge Node Pod includes a built-in log collection sidecar container (oredge-node-log-collector) based on Fluent Bit. This sidecar container automatically collects various log files produced by OpenResty Edge Node and outputs them to the container’s standard output (stdout) and standard error (stderr), seamlessly integrating with the Kubernetes logging ecosystem. You can view and search these logs using the kubectl logs command or through log platforms deployed in your cluster (such as Datadog, ELK, Loki, etc.).

Collected Log Types

The log collector covers the following OpenResty Edge Node log files:

Log Category	Log File	Output Target
HTTP Application Access Log	`access.log`	stdout
HTTP Application Error Log	`error.log`	stderr
Stream Access Log	`stream_access.log`	stdout
Stream Error Log	`stream_error.log`	stderr
DNS Access Log	`dns_access.log`	stdout
DNS Error Log	`dns_error.log`	stderr
HTTP Forward Proxy Access Log	`http_proxy_access.log`	stdout
HTTP Forward Proxy Error Log	`http_proxy_error.log`	stderr
SOCKS5 Access Log	`socks5_access.log`	stdout
SOCKS5 Error Log	`socks5_error.log`	stderr
CC Protection Log	`cc.log`	stderr
WAF Protection Log	`waf.log`	stderr

All access logs are output to stdout, while all error logs and security protection logs (CC, WAF) are output to stderr.

Viewing Logs

You can use the kubectl logs command to view the log collector’s output:

# View all logs (stdout + stderr)
kubectl logs <oredge-node-pod-name> -c oredge-node-log-collector -n openresty-edge
# View only the last 100 lines
kubectl logs <oredge-node-pod-name> -c oredge-node-log-collector -n openresty-edge --tail=100
# Follow new logs in real time
kubectl logs <oredge-node-pod-name> -c oredge-node-log-collector -n openresty-edge -f

Each log entry is automatically prefixed with the source file name in the format <filename>: <log content>, making it easy to distinguish between different log sources. For example:

access.log: 192.168.1.1 - - [23/Mar/2026:10:00:00 +0800] "GET / HTTP/1.1" 200 ...
error.log: 2026/03/23 10:00:01 [error] 12345#0: *1 connect() failed ...
waf.log: 2026/03/23 10:00:02 [waf] rule 1001 triggered ...

Restrictions on Log File Names and Paths

Note: In the Kubernetes deployment mode, the log file names and log paths of OpenResty Edge Node are fixed configurations and cannot be customized through OpenResty Edge Admin. The log collector relies on these fixed file paths for collection. If the log file names or paths are modified, the log collector will be unable to collect the corresponding logs, and those logs will not appear in the container’s standard output.

Log Collector Resource Configuration

The default resource configuration for the log collection sidecar container is as follows:

Resource Type	Requests	Limits
CPU	50m	500m
Memory	64Mi	512Mi

This configuration is suitable for most scenarios. If your OpenResty Edge Node produces a particularly high volume of logs, you can increase the resource limits accordingly in the deployment file.

You can check the actual resource usage of the log collector using the following command (requires Metrics Server to be deployed in the cluster):

kubectl top pod <oredge-node-pod-name> -n openresty-edge --containers

Example output:

$ kubectl top pod oredge-node-0 -n openresty-edge --containers
POD NAME CPU(cores) MEMORY(bytes)
oredge-node-0 oredge-node 7m 279Mi
oredge-node-0 oredge-node-log-collector 1m 31Mi

OpenResty Inc. Documentation – OpenResty Edge™ Log Collection

Edge: OpenResty Edge™ Log Collection (Host Deployment)

# Filebeat Installation

# Filebeat Configuration

# Validation

Edge: OpenResty Edge™ Log Collection (Kubernetes Deployment)

# Collected Log Types

# Viewing Logs

# Restrictions on Log File Names and Paths