Watch Nomad allocations and update logshipper config

Nomad Logger

This is a simple Go application that polls the Nomad API for all allocations on a certain host, and then updates a promtail file_sd_config file, so promtail can scrape and ship logs.

Usage

Usage: nomad-logger [--nomad-address NOMAD-ADDRESS] [--nomad-allocs-dir NOMAD-ALLOCS-DIR] [--nomad-node-id NOMAD-NODE-ID] [--promtail-targets-file PROMTAIL-TARGETS-FILE]

Options:
  --nomad-address NOMAD-ADDRESS
                         The address of the Nomad API [default: http://localhost:4646, env: NOMAD_ADDRESS]
  --nomad-allocs-dir NOMAD-ALLOCS-DIR
                         The location of the Nomad allocations data. Used to set the path to the logfiles [default: /var/lib/nomad/alloc, env: NOMAD_ALLOCS_DIR]
  --nomad-node-id NOMAD-NODE-ID
                         The ID of the Nomad node to collect logs for. If empty, we'll suppose this also runs in as a nomad job, and the available env vars will be used to determine the Node ID [env: NOMAD_NODE_ID]
  --promtail-targets-file PROMTAIL-TARGETS-FILE
                         The promtail file_sd_config file where the generated config can be written. Will be completely overwritten, so don't put anything else there. [default: /etc/promtail/nomad.yaml, env: PROMTAIL_TARGETS_FILE]
  --help, -h             display this help and exit

Example

There is an example nomad job file in examples/nomad-job.hcl. If this is the only job you are running on that node, than the resulting promtail file_sd_config file will look something like examples/promtail-nomad.yaml.

Installing/Building

You can just go install github.com/dmaes/nomad-logger@latest or git clone and go build this.

Comments
  • Consider changing NOMAD_ADDRESS to NOMAD_ADDR

    Consider changing NOMAD_ADDRESS to NOMAD_ADDR

    Mainly since the Nomad CLI uses NOMAD_ADDR as well as most tooling in the ecosystem: https://www.nomadproject.io/docs/commands#nomad_addr

    ...or maybe we can support both NOMAD_ADDRESS and NOMAD_ADDR?

    Thanks for developing a great tool!

  • Update docker/setup-qemu-action action to v2

    Update docker/setup-qemu-action action to v2

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | docker/setup-qemu-action | action | major | v1 -> v2 |


    Release Notes

    docker/setup-qemu-action

    v2

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update docker/setup-buildx-action action to v2

    Update docker/setup-buildx-action action to v2

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | docker/setup-buildx-action | action | major | v1 -> v2 |


    Release Notes

    docker/setup-buildx-action

    v2

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update docker/login-action action to v2

    Update docker/login-action action to v2

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | docker/login-action | action | major | v1 -> v2 |


    Release Notes

    docker/login-action

    v2

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update docker/build-push-action action to v3

    Update docker/build-push-action action to v3

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | docker/build-push-action | action | major | v2 -> v3 |


    Release Notes

    docker/build-push-action

    v3

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update actions/checkout action to v3

    Update actions/checkout action to v3

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | actions/checkout | action | major | v2 -> v3 |


    Release Notes

    actions/checkout

    v3

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update actions/cache action to v3

    Update actions/cache action to v3

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | actions/cache | action | major | v2 -> v3 |


    Release Notes

    actions/cache

    v3

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module go to 1.19

    Update module go to 1.19

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | go (source) | golang | minor | 1.16 -> 1.19 |


    Release Notes

    golang/go

    v1.19.4

    v1.19.3

    v1.19.2

    v1.19.1

    v1.19.0

    v1.18.9

    v1.18.8

    v1.18.7

    v1.18.6

    v1.18.5

    v1.18.4

    v1.18.3

    v1.18.2

    v1.18.1

    v1.18.0

    v1.17.13

    v1.17.12

    v1.17.11

    v1.17.10

    v1.17.9

    v1.17.8

    v1.17.7

    v1.17.6

    v1.17.5

    v1.17.4

    v1.17.3

    v1.17.2

    v1.17.1

    v1.17.0


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module gopkg.in/yaml.v3 to v3.0.1

    Update module gopkg.in/yaml.v3 to v3.0.1

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | gopkg.in/yaml.v3 | require | patch | v3.0.0-20210107192922-496545a6307b -> v3.0.1 |


    Release Notes

    go-yaml/yaml

    v3.0.1

    Compare Source


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module github.com/alexflint/go-arg to v1.4.3

    Update module github.com/alexflint/go-arg to v1.4.3

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | github.com/alexflint/go-arg | require | patch | v1.4.2 -> v1.4.3 |


    Release Notes

    alexflint/go-arg

    v1.4.3: Version 1.4.3

    Compare Source

    • Correctly brackets non-required positional arguments in usage strings
    • Fixes help and usage strings for subcommands.
    • Add native support for url.URL via go-scalar
    • Add environment variables names to error messages when relevant

    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Configure Renovate

    Configure Renovate

    Mend Renovate

    Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

    ๐Ÿšฆ To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.


    Detected Package Files

    • .github/workflows/container.yml (github-actions)
    • go.mod (gomod)

    Configuration Summary

    Based on the default config's presets, Renovate will:

    • Start dependency updates only once this onboarding PR is merged
    • Enable Renovate Dependency Dashboard creation.
    • If Renovate detects semantic commits, it will use semantic commit type fix for dependencies and chore for all others.
    • Ignore node_modules, bower_components, vendor and various test/tests directories.
    • Autodetect whether to pin dependencies or maintain ranges.
    • Rate limit PR creation to a maximum of two per hour.
    • Limit to maximum 10 open PRs at any time.
    • Group known monorepo packages together.
    • Use curated list of recommended non-monorepo package groupings.
    • A collection of workarounds for known problems with packages.

    ๐Ÿ”ก Would you like to change the way Renovate is upgrading your dependencies? Simply edit the renovate.json in this branch with your custom config and the list of Pull Requests in the "What to Expect" section below will be updated the next time Renovate runs.


    What to Expect

    With your current configuration, Renovate will create 10 Pull Requests:

    Update github.com/hashicorp/nomad/api digest to 3afe191
    • Schedule: ["at any time"]
    • Branch name: renovate/github.com-hashicorp-nomad-api-digest
    • Merge into: main
    • Upgrade github.com/hashicorp/nomad/api to 3afe191c9f7c37d03b91d4f80238b8cb95cd9999
    Update module github.com/alexflint/go-arg to v1.4.3
    • Schedule: ["at any time"]
    • Branch name: renovate/github.com-alexflint-go-arg-1.x
    • Merge into: main
    • Upgrade github.com/alexflint/go-arg to v1.4.3
    Update module gopkg.in/yaml.v3 to v3.0.1
    • Schedule: ["at any time"]
    • Branch name: renovate/gopkg.in-yaml.v3-3.x
    • Merge into: main
    • Upgrade gopkg.in/yaml.v3 to f6f7691b1fdeb513f56608cd2c32c51f8194bf51
    Update module go to 1.19
    • Schedule: ["at any time"]
    • Branch name: renovate/go-1.x
    • Merge into: main
    • Upgrade go to 1.19
    Update actions/cache action to v3
    • Schedule: ["at any time"]
    • Branch name: renovate/actions-cache-3.x
    • Merge into: main
    • Upgrade actions/cache to v3
    Update actions/checkout action to v3
    • Schedule: ["at any time"]
    • Branch name: renovate/actions-checkout-3.x
    • Merge into: main
    • Upgrade actions/checkout to v3
    Update docker/build-push-action action to v3
    • Schedule: ["at any time"]
    • Branch name: renovate/docker-build-push-action-3.x
    • Merge into: main
    • Upgrade docker/build-push-action to v3
    Update docker/login-action action to v2
    • Schedule: ["at any time"]
    • Branch name: renovate/docker-login-action-2.x
    • Merge into: main
    • Upgrade docker/login-action to v2
    Update docker/setup-buildx-action action to v2
    • Schedule: ["at any time"]
    • Branch name: renovate/docker-setup-buildx-action-2.x
    • Merge into: main
    • Upgrade docker/setup-buildx-action to v2
    Update docker/setup-qemu-action action to v2
    • Schedule: ["at any time"]
    • Branch name: renovate/docker-setup-qemu-action-2.x
    • Merge into: main
    • Upgrade docker/setup-qemu-action to v2

    ๐Ÿšธ Branch creation will be limited to maximum 2 per hour, so it doesn't swamp any CI resources or overwhelm the project. See docs for prhourlylimit for details.


    โ“ Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.


    This PR has been generated by Mend Renovate. View repository job log here.

  • Dependency Dashboard

    Dependency Dashboard

    This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

    Open

    These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

    Detected dependencies

    github-actions
    .github/workflows/container.yml
    • actions/checkout v3
    • docker/setup-qemu-action v2
    • docker/setup-buildx-action v2
    • actions/cache v3
    • docker/login-action v2
    • docker/build-push-action v3
    gomod
    go.mod
    • go 1.19
    • github.com/alexflint/go-arg v1.4.3
    • github.com/hashicorp/nomad/api v0.0.0-20230103221135-ce00d683f9be@ce00d683f9be
    • gopkg.in/yaml.v3 v3.0.1

    • [ ] Check this box to trigger a request for Renovate to run again on this repository
  • Update github.com/hashicorp/nomad/api digest to ce00d68

    Update github.com/hashicorp/nomad/api digest to ce00d68

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | github.com/hashicorp/nomad/api | require | digest | 86e2dd7 -> ce00d68 |


    Configuration

    ๐Ÿ“… Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    ๐Ÿšฆ Automerge: Disabled by config. Please merge this manually once you are satisfied.

    โ™ป Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    ๐Ÿ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

ChangeTower is intended to help you watch changes in webpages and get notified of any changes written in Go

ChangeTower is intended to help you watch changes in webpages and get notified of any changes written in Go

Nov 17, 2022
Package zaperations provides a Google Cloud operations suite (formerly Stackdriver) compatible config for the uber-go/zap logger.

Package zaperations provides a Google Cloud Operations (formerly Stackdriver) compatible config for the excellent uber-go/zap logger. Example This exa

Nov 6, 2021
Fast, zero config web endpoint change monitor
Fast, zero config web endpoint change monitor

web monitor fast, zero config web endpoint change monitor. for comparing responses, a selected list of http headers and the full response body is stor

Nov 17, 2022
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

The open-source platform for monitoring and observability. Grafana allows you to query, visualize, alert on and understand your metrics no matter wher

Jan 3, 2023
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.
Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Gowl is a process management and process monitoring tool at once. An infinite worker pool gives you the ability to control the pool and processes and monitor their status.

Nov 10, 2022
Simple and configurable Logging in Go, with level, formatters and writers

go-log Logging package similar to log4j for the Golang. Support dynamic log level Support customized formatter TextFormatter JSONFormatter Support mul

Sep 26, 2022
A Go (golang) package providing high-performance asynchronous logging, message filtering by severity and category, and multiple message targets.

ozzo-log Other languages ็ฎ€ไฝ“ไธญๆ–‡ ะ ัƒััะบะธะน Description ozzo-log is a Go package providing enhanced logging support for Go programs. It has the following fe

Dec 17, 2022
Library and program to parse and forward HAProxy logs

haminer Library and program to parse and forward HAProxy logs. Supported forwarder, Influxdb Requirements Go for building from source code git for dow

Aug 17, 2022
Cloudinsight Agent is a system tool that monitors system processes and services, and sends information back to your Cloudinsight account.

Cloudinsight Agent ไธญๆ–‡็‰ˆ README Cloudinsight Agent is written in Go for collecting metrics from the system it's running on, or from other services, and

Nov 3, 2022
Distributed simple and robust release management and monitoring system.
Distributed simple and robust release management and monitoring system.

Agente Distributed simple and robust release management and monitoring system. **This project on going work. Road map Core system First worker agent M

Nov 17, 2022
List files and their creation, modification and access time on android

andfind List files and their access, modification and creation date on a Android

Jan 5, 2022
Every 10 minutes, memory, cpu and storage usage is checked and if they over 80%, sending alert via email.

linux-alert Every 10 minutes, memory, cpu and storage usage is checked and if they over 80%, sending alert via email. Usage Create .env file from .env

Feb 6, 2022
Some tests and examples with goroutines and channels

goroutine-playground Some tests and examples with goroutines and channels simpleAsyncCalls Runs functions in background and doesn't wait for results a

Feb 9, 2022
Golog is a logger which support tracing and other custom behaviors out of the box. Blazing fast and simple to use.

GOLOG Golog is an opinionated Go logger with simple APIs and configurable behavior. Why another logger? Golog is designed to address mainly two issues

Oct 2, 2022
Pragmatic and minimalistic module for collecting and sending traces from Go code ๐Ÿ’ช๐Ÿฝ
Pragmatic and minimalistic module for collecting and sending traces from Go code ๐Ÿ’ช๐Ÿฝ

tracing-go Pragmatic and minimalistic module for collecting and exporting trace data from the Go code. prometheus/client_golang but for Traces NOTE: T

Jan 6, 2023
Simple and blazing fast lockfree logging library for golang
Simple and blazing fast lockfree logging library for golang

glg is simple golang logging library Requirement Go 1.11 Installation go get github.com/kpango/glg Example package main import ( "net/http" "time"

Nov 28, 2022
a golang log lib supports level and multi handlers

go-log a golang log lib supports level and multi handlers Use import "github.com/siddontang/go-log/log" //log with different level log.Info("hello wo

Dec 29, 2022
The Simplest and worst logging library ever written

gologger A Simple Easy to use go logger library. Displays Colored log into console in any unix or windows platform. You can even store your logs in fi

Sep 26, 2022
Simple, configurable and scalable Structured Logging for Go.

log Log is a simple, highly configurable, Structured Logging library Why another logging library? There's allot of great stuff out there, but also tho

Sep 26, 2022