WIP Go Thing to download HCP Vault Logs

Example Go Script to pull HCP Vault Audit Logs

WARNING: This makes use of unstable preview APIs which could change at any time!

USE AT YOUR OWN PERIL

Why?

HCP Vault has audit logs! Woo!

HCP Vault only supports streaming those to a small number of places! Boo!

HCP Vault also only supports downloading audit logs through the HCP Portal manually! Double boo!

Fortunately, there is an official API on the way for this.

In the short term, we can use the preview API

This script uses https://github.com/hashicorp/hcp-sdk-go to interact with that.

Requirements

Environment variables to define the cluster:

export HCP_ORGANIZATION_ID=<YOUR HCP ORG HERE>
export HCP_PROJECT_ID=<YOUR HCP PROJECT HERE>
export HCP_VAULT_CLUSTER_ID=vault-cluster

Environment variables for HCP Auth:

export HCP_CLIENT_ID=<YOUR CLIENT ID HERE>
export HCP_CLIENT_SECRET=<YOUR CLIENT SECRET HERE>

Running

go run main.go

Example output:

$ go run main.go
Response: &models.HashicorpCloudVault20201125Cluster{Config:(*models.HashicorpCloudVault20201125ClusterConfig)(0xc0004acfc0), CreatedAt:strfmt.DateTime{wall:0x1464cf58, ext:63773966983, loc:(*time.Location)(nil)}, CurrentVersion:"v1.8.5", DNSNames:(*models.HashicorpCloudVault20201125ClusterDNSNames)(0xc0000e2040), ID:"vault-cluster", Location:(*models.HashicorpCloudLocationLocation)(0xc0000bb9e0), State:"RUNNING"}

Response: "a6c7a6d5-0fa5-4193-9647-ad70f13f34da"

State: PENDING
State: CREATING
State: READYResponse: &models.HashicorpCloudVault20201125AuditLog{ClusterID:"vault-cluster", DownloadURL:"https://hcp-data-plane-blob-prod.s3.amazonaws.com/225af347-0fd9-41b1-8571-bed0d3ef665e/auditlogs/98b8bfa8-3115-4831-a598-434d02f83786?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZDKPXWD4FHMAK4HM%2F20211201%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211201T160154Z&X-Amz-Expires=900&X-Amz-Security-Token=FwoGZXIvYXdzEFEaDIKka9XFbRoODVjZSiLjAVJ4U5tMj8mqcuc9c2hzVpvCy6jj2TtLieIkyjTRbFOuqWmLPgdjMkg56tLc7dQBrxBwG3qjXxVQQ%2Fll%2BCsANoiLH6WoHhBZUYz3S0SthOnYSU5E66oTUhpjOMnglKud4drTDYKR2Ljgvpjz0sAmS0Ynko9CVHRwbJIAefpbj4p0MKnVCI6IzmGazfNJxsvIO8EFtG7UbVjDf2tcKcm90oroU0W3tdoHdA6NE6JXM2AuN3M4vsMQfHCqLtjUJ%2F6xxIL3n4yTH6SulQex4IXXbMSzZC5z04c8ZLVrbcs8UhFTZhDIKNGono0GMi2kWNkUFLqdPY3%2FhdLOKeKv%2FTNjrqVW5FSWHQtdJ3LMrTAKfV9jmBRorwPKnFY%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=246779b825d4f1bd9b410bb8413a48444a22c04f690b1a6dc2f99cf2de945c41", ExpiresAt:strfmt.DateTime{wall:0x2c99e860, ext:63773971304, loc:(*time.Location)(nil)}, FinishedAt:strfmt.DateTime{wall:0x2c99e860, ext:63773971304, loc:(*time.Location)(nil)}, ID:"a6c7a6d5-0fa5-4193-9647-ad70f13f34da", IntervalEnd:strfmt.DateTime{wall:0xa7d8c0, ext:63773971294, loc:(*time.Location)(nil)}, IntervalStart:strfmt.DateTime{wall:0xa7d8c0, ext:63773969494, loc:(*time.Location)(nil)}, Location:(*models.HashicorpCloudLocationLocation)(0xc00016a450), State:"READY"}

Download URL: "https://hcp-data-plane-blob-prod.s3.amazonaws.com/225af347-0fd9-41b1-8571-bed0d3ef665e/auditlogs/98b8bfa8-3115-4831-a598-434d02f83786?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAZDKPXWD4FHMAK4HM%2F20211201%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211201T160154Z&X-Amz-Expires=900&X-Amz-Security-Token=FwoGZXIvYXdzEFEaDIKka9XFbRoODVjZSiLjAVJ4U5tMj8mqcuc9c2hzVpvCy6jj2TtLieIkyjTRbFOuqWmLPgdjMkg56tLc7dQBrxBwG3qjXxVQQ%2Fll%2BCsANoiLH6WoHhBZUYz3S0SthOnYSU5E66oTUhpjOMnglKud4drTDYKR2Ljgvpjz0sAmS0Ynko9CVHRwbJIAefpbj4p0MKnVCI6IzmGazfNJxsvIO8EFtG7UbVjDf2tcKcm90oroU0W3tdoHdA6NE6JXM2AuN3M4vsMQfHCqLtjUJ%2F6xxIL3n4yTH6SulQex4IXXbMSzZC5z04c8ZLVrbcs8UhFTZhDIKNGono0GMi2kWNkUFLqdPY3%2FhdLOKeKv%2FTNjrqVW5FSWHQtdJ3LMrTAKfV9jmBRorwPKnFY%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=246779b825d4f1bd9b410bb8413a48444a22c04f690b1a6dc2f99cf2de945c41"

You can then download from that URL, gunzip it, then do whatever you like with your audit logs

Owner
Strawb's HashiCorp Demos
Repos which we use for demos. We make no guarantee that these will be up-to-date or working, unless we specifically send you one
Strawb's HashiCorp Demos
Similar Resources

A customized GORM logger that implements the appropriate interface and uses Logrus to output logs

CryptoMath GORM Logger A customized GORM logger that implements the appropriate interface and uses Logrus to output logs. Install go get github.com/ma

Nov 6, 2021

Lumberjack is a Go package for writing logs to rolling files.

Lumberjack is a Go package for writing logs to rolling files.

Feb 24, 2022

This POC is built with the goal to collect events/logs from the host systems such as Kubernetes, Docker, VMs, etc. A buffering layer is added to buffer events from the collector

This POC is built with the goal to collect events/logs from the host systems such as Kubernetes, Docker, VMs, etc. A buffering layer is added to buffer events from the collector

What is does This POC is build with the goal to collect events/logs from the host systems such as Kubernetes, docker, VMs etc. A buffering layer is ad

Nov 11, 2022

Leveled execution logs for Go.

glog Leveled execution logs for Go. This is an efficient pure Go implementation of leveled logs in the manner of the open source C++ package glog. By

Nov 29, 2021

Stream logs through websockets, written in Go

Stream logs through websockets, written in Go

Jan 8, 2022

Request-logging-tool - A tool logs the md5 codes of the responses of the given domains in parameter

request-logging-tool Application to send http requests and log the md5 responses

Jan 7, 2022

Log-generator - A simple CLI tool that generates near real logs for testing

Log-generator - A simple CLI tool that generates near real logs for testing

Jan 22, 2022

A logrus.Hook that logs with a zap.Logger

zaprus Ever had a 3rd-party dependency requiring a logrus, but you're using zap? zaprus provides a logrus.Hook that makes a logrus.(Entry|Logger) repl

Feb 27, 2022

Logger - Some implementations for Logs written in Go

Logger will be a tool for sending logs to different places in your software. Rig

Feb 16, 2022
Leveled execution logs for Go

glog ==== Leveled execution logs for Go. This is an efficient pure Go implementation of leveled logs in the manner of the open source C++ package h

Dec 24, 2022
Library and program to parse and forward HAProxy logs

haminer Library and program to parse and forward HAProxy logs. Supported forwarder, Influxdb Requirements Go for building from source code git for dow

Aug 17, 2022
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

The open-source platform for monitoring and observability. Grafana allows you to query, visualize, alert on and understand your metrics no matter wher

Jan 3, 2023
raft variant with topology order logs

Safe: A log that is safe if it has been replicated to a quorum, no matter whether or not the committed flag is set on any replica.

May 28, 2022
Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.
Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console.

logkit-community 中文版 Introduce Very powerful server agent for collecting & sending logs & metrics with an easy-to-use web console. logkit-community De

Dec 29, 2022
Like Prometheus, but for logs.
Like Prometheus, but for logs.

Loki: like Prometheus, but for logs. Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It

Dec 30, 2022
Pixie gives you instant visibility by giving access to metrics, events, traces and logs without changing code.
Pixie gives you instant visibility by giving access to metrics, events, traces and logs without changing code.

Pixie gives you instant visibility by giving access to metrics, events, traces and logs without changing code.

Jan 4, 2023
Search and analysis tooling for structured logs

Zed The Zed system provides an open-source, cloud-native, and searchable data lake for semi-structured and structured data. Zed lakes utilize a supers

Jan 5, 2023
gtl - Gemini Tiny Logs - A simple TUI for the tinylog format on gemini
gtl - Gemini Tiny Logs - A simple TUI for the tinylog format on gemini

GTL: Gemini Tiny Logs Goal: A TUI for the tinylogs format on the gemini space. See screenshots Installation gtl requires go ≥ 1.16 From Source git clo

Dec 1, 2022
mtail - extract internal monitoring data from application logs for collection into a timeseries database
 mtail - extract internal monitoring data from application logs for collection into a timeseries database

mtail - extract internal monitoring data from application logs for collection into a timeseries database mtail is a tool for extracting metrics from a

Dec 29, 2022