ECS task event/log tracer CLI

tracer

tracer is a tracing tool for Amazon ECS tasks.

tracer shows events and logs of the tasks order by timestamp.

example

Run a task successfully and shutdown.

$ tracer default 84b5991528504856a2f003b7da9b2b82
2021-11-27T01:49:41.251+09:00   TASK    Created
2021-11-27T01:49:55.744+09:00   TASK    Pull started
2021-11-27T01:50:02.269+09:00   TASK    Pull stopped
2021-11-27T01:50:03.112+09:00   CONTAINER:nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
2021-11-27T01:50:03.112+09:00   CONTAINER:nginx /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
2021-11-27T01:50:03.112+09:00   CONTAINER:nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
2021-11-27T01:50:03.123+09:00   CONTAINER:nginx 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
2021-11-27T01:50:03.125+09:00   CONTAINER:nginx 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
2021-11-27T01:50:03.125+09:00   CONTAINER:nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
2021-11-27T01:50:03.128+09:00   CONTAINER:nginx /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
2021-11-27T01:50:03.131+09:00   CONTAINER:nginx /docker-entrypoint.sh: Configuration complete; ready for start up
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: using the "epoll" event method
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: nginx/1.21.4
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6) 
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: OS: Linux 4.14.248-189.473.amzn2.aarch64
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1024:4096
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: start worker processes
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: start worker process 34
2021-11-27T01:50:03.135+09:00   CONTAINER:nginx 2021/11/26 16:50:03 [notice] 1#1: start worker process 35
2021-11-27T01:50:03.154+09:00   TASK    Started
2021-11-27T02:01:58.598+09:00   TASK    Stopping
2021-11-27T02:01:58.598+09:00   TASK    StoppedReason:Request stop task by user action.
2021-11-27T02:01:58.598+09:00   TASK    StoppedCode:UserInitiated
2021-11-27T02:01:59.129+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: signal 15 (SIGTERM) received, exiting
2021-11-27T02:01:59.129+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 34#34: exiting
2021-11-27T02:01:59.129+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 34#34: exit
2021-11-27T02:01:59.129+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 35#35: exiting
2021-11-27T02:01:59.129+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 35#35: exit
2021-11-27T02:01:59.179+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: signal 14 (SIGALRM) received
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: signal 17 (SIGCHLD) received from 35
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: worker process 35 exited with code 0
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: signal 29 (SIGIO) received
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: signal 17 (SIGCHLD) received from 34
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: worker process 34 exited with code 0
2021-11-27T02:01:59.215+09:00   CONTAINER:nginx 2021/11/26 17:01:59 [notice] 1#1: exit
2021-11-27T02:02:10.402+09:00   TASK    Execution stopped
2021-11-27T02:02:33.559+09:00   TASK    Stopped

Failed to run task. (typo container image URL)

$ tracer default 9f654c76cde14c7c85cf54dce087658a
2021-11-27T02:29:15.055+09:00   TASK    Created
2021-11-27T02:29:33.527+09:00   TASK    Execution stopped
2021-11-27T02:29:43.569+09:00   TASK    Stopping
2021-11-27T02:29:43.569+09:00   TASK    StoppedReason:CannotPullContainerError: inspect image has been retried 1 time(s): failed to resolve ref "docker.io/library/ngin:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
2021-11-27T02:29:43.569+09:00   TASK    StoppedCode:TaskFailedToStart
2021-11-27T02:29:57.070+09:00   TASK    Stopped
Owner
FUJIWARA Shunichiro
FUJIWARA Shunichiro
https://github.com/fujiwara/tracer
Similar Resources

lumberjack is a log rolling package for Go

lumberjack Lumberjack is a Go package for writing logs to rolling files. Package lumberjack provides a rolling logger. Note that this is v2.0 of lumbe

Jan 1, 2023

CoLog is a prefix-based leveled execution log for Go

CoLog is a prefix-based leveled execution log for Go

What's CoLog? CoLog is a prefix-based leveled execution log for Go. It's heavily inspired by Logrus and aims to offer similar features by parsing the

Dec 14, 2022

OpenTelemetry log collection library

opentelemetry-log-collection Status This project was originally developed by observIQ under the name Stanza. It has been contributed to the OpenTeleme

Sep 15, 2022

A simple web service for storing text log files

logpaste A minimalist web service for uploading and sharing log files. Run locally go run main.go Run in local Docker container The Docker container a

Dec 30, 2022

exo: a process manager & log viewer for dev

exo: a process manager & log viewer for dev

exo: a process manager & log viewer for dev exo- prefix – external; from outside. Features Procfile compatible process manager.

Dec 28, 2022

Write log entries, get X-Ray traces.

logtoxray Write to logs, get X-Ray traces. No distributed tracing instrumenation library required. 🚧 🚧 🚧 THIS PROJECT IS A WORK-IN-PROGRESS PROTOTY

Apr 24, 2022

Binalyze logger is an easily customizable wrapper for logrus with log rotation

logger logger is an easily customizable wrapper for logrus with log rotation Usage There is only one function to initialize logger. logger.Init() When

Oct 2, 2022

Log-structured virtual disk in Ceph

Log-structured virtual disk in Ceph

lsd_ceph Log-structured virtual disk in Ceph 1. Vision and Goals of the Project Implement the basic librbd API to work with the research block device

Dec 13, 2021

Multi-level logger based on go std log

mlog the mlog is multi-level logger based on go std log. It is: Simple Easy to use NOTHING ELSE package main import ( log "github.com/ccpaging/lo

May 18, 2022
Comments
  • When executing from a lambda function, the SNS subject is too long when using ARNs

    When executing from a lambda function, the SNS subject is too long when using ARNs

    thank you great tool.

    When executing from a lambda function, the SNS subject is too long when using ARNs

    for example:

    Tracer: arn:aws:ecs:ap-northeast-1:012345678901:task/dev-main/0123456789abcdef0123456789abcdef on arn:aws:ecs:ap-northeast-1:012345678901:cluster/main
2022-03-23T03:34:40.757Z	TASK	Created
2022-03-23T03:34:44.431Z	TASK	Connected
2022-03-23T03:34:58.254Z	TASK	Pull started
2022-03-23T03:35:18.965Z	TASK	Pull stopped
2022-03-23T03:35:37.090Z	TASK	Started
2022-03-23T03:41:15.397Z	TASK	Execution stopped
2022-03-23T03:41:25.441Z	TASK	Stopping
2022-03-23T03:41:25.441Z	TASK	StoppedReason:Essential container in task exited
2022-03-23T03:41:25.441Z	TASK	StoppedCode:EssentialContainerExited
2022-03-23T03:41:38.629Z	TASK	Stopped
2022-03-23T03:44:32.977Z	CONTAINER:hoge	LastStatus:STOPPED HealthStatus:UNKNOWN (exit code: 0)
2022-03-23T03:44:32.977Z	TASK	LastStatus:STOPPED
InvalidParameter: Invalid parameter: Subject

    from aws cli help message:

    
       --subject (string)
          Optional parameter to be used as the "Subject" line when the message
          is  delivered  to email endpoints. This field will also be included,
          if present, in the standard JSON messages delivered  to  other  end-
          points.

          Constraints:  Subjects must be ASCII text that begins with a letter,
          number, or punctuation mark; must not include line breaks or control
          characters; and must be less than 100 characters long.

    Sending SNS fails because the subject is too long. Therefore, we have made two changes.

    1. cluster name and task ID are extracted from ARN in the Lambda handler.
    2. if the Subject is longer than 100 characters, omit the end
  • Create the flag to display the version

    Create the flag to display the version

    About

    It will be more helpful if the command provides a way to show the version of itself. To adjust the style as you prefer, I referred to the repositories below when writing code!

    • https://github.com/fujiwara/stretcher
    • https://github.com/kayac/ecspresso
  • Remove `time.Time` fields from the type of input payload of lambda handler function.

    Remove `time.Time` fields from the type of input payload of lambda handler function.

    Thank you for the awesome tool ❤️

    What This PR do?

    Remove time.Time fields from the type of input payload of lambda handler function.

    Why?

    To make it handle payload typed with Task, like results of ECS tasks running on AWS Step Functions.

    Context

    On AWS Step Functions, I'd like to collect error log from ECS Tasks with tracer deployed as a lambda function.

    image

    And I got error like below.

    {
    "errorMessage": "parsing time \"1661779180572\" as \"\\\"2006-01-02T15:04:05Z07:00\\\"\": cannot parse \"1661779180572\" as \"\\\"\"",
    "errorType": "ParseError"
}

    It's because tracer just expects payload from ECS events and tries to handle unixtime fields as RFC3339.

    This issue is noted as

    The values for the createdAt, connectivityAt, pullStartedAt, startedAt, pullStoppedAt, and updatedAt fields are UNIX timestamps in the response of a DescribeTasks action whereas in the task state change event they are ISO string timestamps.

    on ECS events

    this patch might be uncool, but I think reasonable because the deleted fields are not referenced. Thanks!

  • show all status of containers and a task.

    show all status of containers and a task.

    e.g.

    2022-03-01T14:39:22.513+09:00   TASK    Stopped
2022-03-01T14:41:08.121+09:00   CONTAINER:web   LastStatus:STOPPED HealthStatus:UNKNOWN (exit code: 1)
2022-03-01T14:41:08.121+09:00   CONTAINER:nginx LastStatus:STOPPED HealthStatus:UNKNOWN
2022-03-01T14:41:08.121+09:00   CONTAINER:envoy LastStatus:STOPPED HealthStatus:UNKNOWN (exit code: 0) (reason: CannotInspectContainerError: Could not transition to inspecting; timed out after waiting 30s)
2022-03-01T14:41:08.121+09:00   TASK    LastStatus:STOPPED
Related tags
Logging and Monitoring tracer
Log-server - Implement log server for gwaylib/log/adapter/rmsq

Implement server of github.com/gwaylib/log Base on https://github.com/gwaycc/lserver Build . env.sh cd cmd/web go build Deploy Install supd(Debian sy

Jan 3, 2022
An golang log lib, supports tracking and level, wrap by standard log lib

Logex An golang log lib, supports tracing and level, wrap by standard log lib How To Get shell go get gopkg.in/logex.v1 source code import "gopkg.in/

Nov 27, 2022
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.
 Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.

Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer, used to analyze Nginx access logs for myself.

Nov 29, 2022
Distributed-Log-Service - Distributed Log Service With Golang
 Distributed-Log-Service - Distributed Log Service With Golang

Distributed Log Service This project is essentially a result of my attempt to un

Jun 1, 2022
Log-analyzer - Log analyzer with golang

Log Analyzer what do we have here? Objective Installation and Running Applicatio

Jan 27, 2022
Goal is to generate logger and tracer wraps around a certain struct
 Goal is to generate logger and tracer wraps around a certain struct

Goal is to generate logger and tracer wraps around a certain struct

Feb 13, 2022
Log-generator - A simple CLI tool that generates near real logs for testing

Log-generator - A simple CLI tool that generates near real logs for testing

Jan 22, 2022
Simple Proof of Concept REST event logger.

REST Event Logger PoC I am working on this project intermittently. I have set myself a time limit of ~3hrs which includes the time to acquire and adap

Feb 10, 2022
a golang log lib supports level and multi handlers

go-log a golang log lib supports level and multi handlers Use import "github.com/siddontang/go-log/log" //log with different level log.Info("hello wo

Dec 29, 2022
Structured log interface

Structured log interface Package log provides the separation of the logging interface from its implementation and decouples the logger backend from yo

Sep 26, 2022