json-log-exporter
A Nginx log parser exporter for prometheus metrics.
Installation
- go mod tidy
- go build
- run binary
Configuration
config/config.yml
- name: app
format: $remote_addr - $remote_user [$time_local] "$method $request $protocol" $request_time-$upstream_response_time $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" $request_id
source_files:
- ./test/nginx.log
external_labels:
region: zone1
relabel_config:
source_labels:
- request
- method
- status
replacement:
request:
trim: "?"
replace:
- target: /v1.0/example/\d+
value: /v1.0/example/:id
histogram_buckets: [0.1, 0.3, 0.5, 1, 2]
exemplar_config:
match:
request_time: ">= 0.3"
labels:
- request_id
- remote_addr
- name: gin
format: $clientip - [$time_local] "$method $request $protocol $status $upstream_response_time "$http_user_agent" $err"
source_files:
- ./test/gin.log
external_labels:
region: zone1
relabel_config:
source_labels:
- request
- method
- status
replacement:
request:
trim: "?"
histogram_buckets: [0.1, 0.3, 0.5, 1, 2]
- name: service name, metric will be
{name}_http_response_count_total,{name}_http_response_count_total,{name}_http_response_size_bytes,{name}_http_upstream_time_seconds,{name}_http_response_time_seconds - source_files: service nginx log, support multiple files.
- external_labels: all metrics will add this labelsets.
- relabel_config:
- source_labels: what's labels should be use.
- replacement: source labelvalue format rule, it supports regrex, eg
/v1.0/example/123?id=q=xxxwill relace to/v1.0/example/:id, it's very powerful.
- histogram_buckets: configure histogram metrics buckets.
- exemplar_config: configure exemplars, it used for histogram metrics.
Example
./test/demo.log
