Nginx log parser and Prometheus exporter
This service parses incoming syslog messages from Nginx sent over UDP and converts them into Prometheus metrics exported through the built-in HTTP server.
-
If the prefix-list is specified then per-URI statistics are generated. Using this feature without URI-prefix list is dangerous because it leads to an unbounded memory usage. It's therefore enabled only with a limited prefix list.
The URI prefix list is a plain-text file with a single prefix per line, e.g.
/api/call1.json /api/call2.json /api/call3.json
URIs received from nginx are stripped of any query parameters - only the part before '?' is used.
-
It optionally supports country lookup for client IPs using MaxMind GeoIP database.
Nginx configuration snippet
log_format collector '$remote_addr|$scheme|$host|$request_method|$server_protocol|$request_uri|$status|$request_time|$request_length|$bytes_sent';
access_log syslog:server=1.1.1.1:1514,tag=nginx collector;
Getting it
-
Get RPM & DEB directly from releases, only linux-amd64 build currently available.
-
Build youself:
go get github.com/blind-oracle/nginx-prometheus
cd $GOPATH/src/github.com/blind-oracle/nginx-prometheus
go build
Usage example
If you're using RPM/DEB packages then options are set in /etc/default/nginx-prometheus
./nginx-prometheus \
-debug false \
-listenSyslog 0.0.0.0:1514 \
-listenHTTP 0.0.0.0:1514 \
-geoipCountryDB /etc/nginx-prometheus/country.mmdb \
-uriPrefixFile /etc/nginx-prometheus/uriPrefixes.txt
- debug - prints every syslog message received if true (defaults to false)
- listenSyslog - ip:port on which to listen for UDP Syslog messages (defaults to 0.0.0.0:1514)
- listenHTTP - ip:port on which to listen for incoming HTTP requests from Prometheus (defaults to 0.0.0.0:11080)
- geoipCountryDB - path to MaxMind country GeoIP database (optional)
- uriPrefixFile - path to a URI prefix list (optional)
Grafana dashboard
Add Prometheus source to Grafana and import grafana-dashboard.json for most common graphs.
