Structured, composable logging for Go

obligatory xkcd

log15 godoc reference Build Status

Package log15 provides an opinionated, simple toolkit for best-practice logging in Go (golang) that is both human and machine readable. It is modeled after the Go standard library's io and net/http packages and is an alternative to the standard library's log package.

Features

  • A simple, easy-to-understand API
  • Promotes structured logging by encouraging use of key/value pairs
  • Child loggers which inherit and add their own private context
  • Lazy evaluation of expensive operations
  • Simple Handler interface allowing for construction of flexible, custom logging configurations with a tiny API.
  • Color terminal support
  • Built-in support for logging to files, streams, syslog, and the network
  • Support for forking records to multiple handlers, buffering records for output, failing over from failed handler writes, + more

Versioning

The API of the master branch of log15 should always be considered unstable. If you want to rely on a stable API, you must vendor the library.

Importing

import log "github.com/inconshreveable/log15"

Examples

// all loggers can have key/value context
srvlog := log.New("module", "app/server")

// all log messages can have key/value context
srvlog.Warn("abnormal conn rate", "rate", curRate, "low", lowRate, "high", highRate)

// child loggers with inherited context
connlog := srvlog.New("raddr", c.RemoteAddr())
connlog.Info("connection open")

// lazy evaluation
connlog.Debug("ping remote", "latency", log.Lazy{pingRemote})

// flexible configuration
srvlog.SetHandler(log.MultiHandler(
    log.StreamHandler(os.Stderr, log.LogfmtFormat()),
    log.LvlFilterHandler(
        log.LvlError,
        log.Must.FileHandler("errors.json", log.JsonFormat()))))

Will result in output that looks like this:

WARN[06-17|21:58:10] abnormal conn rate                       module=app/server rate=0.500 low=0.100 high=0.800
INFO[06-17|21:58:10] connection open                          module=app/server raddr=10.0.0.1

Breaking API Changes

The following commits broke API stability. This reference is intended to help you understand the consequences of updating to a newer version of log15.

  • 57a084d014d4150152b19e4e531399a7145d1540 - Added a Get() method to the Logger interface to retrieve the current handler
  • 93404652ee366648fa622b64d1e2b67d75a3094a - Record field Call changed to stack.Call with switch to github.com/go-stack/stack
  • a5e7613673c73281f58e15a87d2cf0cf111e8152 - Restored syslog.Priority argument to the SyslogXxx handler constructors

FAQ

The varargs style is brittle and error prone! Can I have type safety please?

Yes. Use log.Ctx:

srvlog := log.New(log.Ctx{"module": "app/server"})
srvlog.Warn("abnormal conn rate", log.Ctx{"rate": curRate, "low": lowRate, "high": highRate})

Regenerating the CONTRIBUTORS file

go get -u github.com/kevinburke/write_mailmap
write_mailmap > CONTRIBUTORS

License

Apache

Comments
  • Add ability to easily log stack traces

    Add ability to easily log stack traces

    Feedback from the mailing list:

    A "stack trace" can be useful sometimes. What about adding a special "stack" key to the log functions (or better, a logger.XxxStack functions) which store the callers in the context.'

    It's a difficult API design problem:

    • Adding new XxxStack functions is too onerous on the API.
    • A simple handler can't just add them because it would end up generating the stack from within the handler tree.
    • Maybe a handler could add them but it would have to do something smart where it walks up the captured stack frames and removes each frame until it reaches the caller of a Logger interface method.

    Most likely, the best option is that this should be a new top-level function that you would use as a context value that we would know how to format nicely for you:

    logger.Error("something went horribly wrong", "stack", log.Stack())
    

    Note that it wouldn't work well with log.Lazy though (you'd end up with frames from the handler tree in the traceback).

    How is it formatted as output? Newline escaping will produce an unreadable mess when looking at log files or on the terminal. grohl does something interesting where it splits the stack into multiple log messages which is interesting to consider:

    t=12412323 lvl=error stack=First line of stack trace
    t=12412323 lvl=error stack=Second line of stack trace
    t=12412323 lvl=error stack=Third line of stack trace
    

    Maybe this format could just be used for the TerminalFormat.

  • tests fail when there is more than one path in $GOPATH

    tests fail when there is more than one path in $GOPATH

    $ GOPATH=$GOPATH:/tmp /usr/bin/go test gopkg.in/inconshreveable/log15.v2/...
    ok      gopkg.in/inconshreveable/log15.v2   0.091s
    ok      gopkg.in/inconshreveable/log15.v2/ext   0.041s
    --- FAIL: TestCallFormat (0.00s)
        stack_test.go:80: fmt.Sprintf("%+s", Call(func)) = gopkg.in/inconshreveable/log15.v2/stack/stack_test.go, want ../../../gopath/src/gopkg.in/inconshreveable/log15.v2/stack/stack_test.go
        stack_test.go:80: fmt.Sprintf("%+v", Call(func)) = gopkg.in/inconshreveable/log15.v2/stack/stack_test.go:24, want ../../../gopath/src/gopkg.in/inconshreveable/log15.v2/stack/stack_test.go:24
        stack_test.go:80: fmt.Sprintf("%+s", Call(meth)) = gopkg.in/inconshreveable/log15.v2/stack/stack_test.go, want ../../../gopath/src/gopkg.in/inconshreveable/log15.v2/stack/stack_test.go
        stack_test.go:80: fmt.Sprintf("%+v", Call(meth)) = gopkg.in/inconshreveable/log15.v2/stack/stack_test.go:18, want ../../../gopath/src/gopkg.in/inconshreveable/log15.v2/stack/stack_test.go:18
    FAIL
    FAIL    gopkg.in/inconshreveable/log15.v2/stack 0.002s
    ?       gopkg.in/inconshreveable/log15.v2/term  [no test files]
    
  • Add file name and line number to context

    Add file name and line number to context

    This turns out to be more tricky than one would hope. Best options that I can think of are:

    1. Change the Handler interface to pass calldepth through and require every implementing handler to appropriately increment it as it goes through the handler tree.
    2. Add another implementation of the Logger interface which includes these parameters in the context. Basically adding a NewSourceInfoLogger call (or something similar).
    3. Change the default logger implementation to always take the performance hit (I haven't measured this yet) of doing runtime.Callers and adding those values as fields in the log.Record. An optional handler could be used to inject them into the context.

    Relevant thread on go-nuts: https://groups.google.com/forum/#!topic/golang-nuts/_EdGGCwY1AI

  • Typed nil pointer in context results in panic

    Typed nil pointer in context results in panic

    If you pass a typed nil pointer to the context,

    Example:

    if player is nil

        log.Warn("Remove issue", log.Ctx{"tier": tier, "player": player, "reason": "player does not exist in cache"})
    

    You will get a panic when the struct's auto generated .String is called using a nil pointer.

    This will happen here:

    format.go:178

    func formatShared(value interface{}) interface{} {
            switch v := value.(type) {
        case time.Time:
            return v.Format(timeFormat)
    
        case error:
            return v.Error()
    
        case fmt.Stringer:
            return v.String()
    
        default:
            return v
        }
    }
    

    This may be a known issue, but I could see accidentally having nil's sent in. From what I have read, to address this, reflect may be needed which would incur an overhead and may not be desired. I thought I'd open this issue to find out if this is deemed an issue. If not, what the best practices are to work around this.

  • Introducing stack pkg with sync.Pool breaks Go 1.2 compatibility

    Introducing stack pkg with sync.Pool breaks Go 1.2 compatibility

    This should probably be mentioned in the Docs. This might also affect users who run on Google App Engine, which AFAIK still runs on Go 1.2.

    Alternatively we could add a build tag to the go files. Speaking of which, I will create a PR ;)

  • (s)printf-style formatting and logging actual errors?

    (s)printf-style formatting and logging actual errors?

    Hi. Are there any plans for providing a shortcut for printf, style formatting or is it idiomatic to write:

    log.Info(fmt.Sprintf("some text: %d > %d", max, min), "someCtx", 42)
    

    It seems sort of verbose, but I could live with it.

    Then another thing, which I tried to look up in the README and the docs but couldn't find. I assume logging error (the Go error type) values happens quite a lot. What is the usual way to deal with this? I have two options, both of which don't really feel that good:

    // #1
    log.Info(err.Error())
    
    // #2
    log.Info("while processing XXX", "error", err)
    

    The first probably doesn't give enough information. The second one feels a bit better but I'm not sure about the key. Should I use "error"?

  • Fix build under appengine

    Fix build under appengine

    Seems like appengine golang currently fails the build because this !appengine isn't being parsed properly. I believe comma-separating the fields is the correct fix there? It functions after this change locally.

  • Don't call escapeString() on time values

    Don't call escapeString() on time values

    escapeString() is very expensive, and we never need to escape time.Time's that have been formatted with timeFormat, so let's return time.Time's early instead of escaping them.

    This saves a lot of allocations:

    benchmark                        old ns/op     new ns/op     delta
    BenchmarkStreamNoCtx-4           4761          4043          -15.08%
    BenchmarkDiscard-4               818           788           -3.67%
    BenchmarkCallerFileHandler-4     1892          1861          -1.64%
    BenchmarkCallerFuncHandler-4     1690          1647          -2.54%
    BenchmarkLogfmtNoCtx-4           3579          2886          -19.36%
    BenchmarkJsonNoCtx-4             1650          1649          -0.06%
    BenchmarkMultiLevelFilter-4      850           833           -2.00%
    BenchmarkDescendant1-4           835           805           -3.59%
    BenchmarkDescendant2-4           858           838           -2.33%
    BenchmarkDescendant4-4           930           894           -3.87%
    BenchmarkDescendant8-4           974           957           -1.75%
    

    It's highly possible there's a better way to do this, maybe have formatShared() return a returnEarly bool or something.

  • failed build on app engine

    failed build on app engine

    Reported failure here: https://github.com/kevinburke/rest/pull/4#issuecomment-277801954

    I've been unable to reproduce it, because you can't specify GOOS=appengine and compile the build. Need to set up the App Engine SDK locally and test using that OS.

    If I had to guess, I would guess that App Engine is not a subset of linux and so the terminal_notwindows.go file also compiles there.

  • log15.Record compatibility across third-party packages

    log15.Record compatibility across third-party packages

    In #51 @ChrisHines wrote:

    [..] It is mildly annoying that one must import log15.v2 in order to implement a Handler (because of the *log15.Record argument). So Handlers are always coupled to log15.v2. This may cause problems down the road for people that implement custom handlers in a library. Consumers of such Handlers will have a harder time upgrading to a putative log15.v3 because I'm pretty sure that all the Handlers in an application must have the same import path for *Record. I'm not sure what to do about that yet.

    Excelent point! Even though the Record structure stays the same, the types will be incompatible because the qualified identifier is different.

    inconshreveable/log15.v2/record One solution would be moving Record to it's own package log15.v2/record. Then log15.v3 will still import log15.v2/record, as will everyone that writes a custom handler. If at v6 we want to make a breaking change the Record type, then log15.v6 will import log15.v6/record, and everyone will have to update their Handlers to work with v6.

    inconshreveable/log15record.v1 The strange thing about the above log15.v2/record subpackage is that if you modify the Record in a non-breaking way (add a field at the bottom of the struct), then those changes must be made in log15.v2, even if the most recent log15 version is actually at v4. And it's also a bit strange that log15.v{3,4,5}/record are never used.. So another solution might be to create a new repo inconshreveable/log15record (package record) that starts at v1, and when breaking changes are made moves to v2..

    In both cases, Lvl and RecordKeyNames must also move to the new package. Ctx doesn't necessarily have to move because on the record the field type for the context is []interface{}, but if log15.Ctx is used by third-party projects (I don't know if that's the case), it might move to record.Ctx as well to solve the same incompatibility problem.

  • Configurable t, lvl, & msg keys

    Configurable t, lvl, & msg keys

    First off, great stuff. I'm glad you put the time in to build such a great log library.

    I was wondering, though, would it be possible to make the time, level, and message keys configurable? Does that seem reasonable?

    https://github.com/inconshreveable/log15/blob/master/format.go#L77 https://github.com/inconshreveable/log15/blob/master/format.go#L121

    I think that would allow for better integration into systems that are expecting different keys for those values.

    Thanks!

  • CallerFileHandler should include package name

    CallerFileHandler should include package name

    Currently if logger is passed to another package it would still show only a file name within that package.

    This may cause issues with finding correct file (there may be identically named files in different packages).

    It can be worked around by chaining CallerFileHandler with CallerFuncHandler. Both are lacking in functionality. CallerFuncHandler will only show function name with package name, but not file line in this function. CallerFileHandler will show file name with line, but not package.

  • Is this library maintained?

    Is this library maintained?

    First of all thanks for creating this library with such a beautiful API

    My concern is that I don't see any new commits in the past couple of years and important PRs like go module addition is not having much of a discussion and is not being merged

    What is the future of this project ?

  • Use semantic versioning to satisfy `go mod`

    Use semantic versioning to satisfy `go mod`

    It appears that go mod does not recognize the current tagging scheme of log15 and always pulls the master. Couls you please use semantic versioning to satisfy it? Changing v2.15 -> v2.15.0 should o the trick. If you could also create v2.15.0 tag in addition to v2.15, that would be great.

  • Modified code to run on PowerSystems

    Modified code to run on PowerSystems

    Hi Here is my contribution to your code, its working good on powersystems.

    Thanks for the code, its working good.

    What do these changes do?

    Added Architecture "ppc64le"

    Are there changes in behavior for the user?

    No

  • format: stricter quoting based on strconv

    format: stricter quoting based on strconv

    This PR replaces the escapeString function with the more stricter quoting done by strconv. Previously, special non-printable characters such as bash escape characters could be allowed through. Such characters can change the terminal colour, break lines, delete previously printed characters etc, and may represent a security vulnerability in the application using this library.

    ~~This PR does change API a bit, since previously log15 did not quote all messages, if they were deemed to not require quoting. It would be possible to revert that change, but I actually think it's better to be consistent, so I didn't add the extra checks to see if the value should be un-quoted again.~~

    This PR now avoids quoting messages that are plain lower range ascii. It does now quote otherwise printable messages with characters like äö, which was not previously quoted.

Gomol is a library for structured, multiple-output logging for Go with extensible logging outputs

gomol Gomol (Go Multi-Output Logger) is an MIT-licensed structured logging library for Go. Gomol grew from a desire to have a structured logging libra

Sep 26, 2022
Structured logging package for Go.
Structured logging package for Go.

Package log implements a simple structured logging API inspired by Logrus, designed with centralization in mind. Read more on Medium. Handlers apexlog

Dec 24, 2022
Simple, configurable and scalable Structured Logging for Go.

log Log is a simple, highly configurable, Structured Logging library Why another logging library? There's allot of great stuff out there, but also tho

Sep 26, 2022
Structured, pluggable logging for Go.
Structured, pluggable logging for Go.

Logrus Logrus is a structured logger for Go (golang), completely API compatible with the standard library logger. Logrus is in maintenance-mode. We wi

Jan 9, 2023
Structured Logging Made Easy
Structured Logging Made Easy

Structured Logging Made Easy Features Dependency Free Simple and Clean Interface Consistent Writer IOWriter, io.Writer wrapper FileWriter, rotating &

Jan 3, 2023
Blazing fast, structured, leveled logging in Go.

⚡ zap Blazing fast, structured, leveled logging in Go. Installation go get -u go.uber.org/zap Note that zap only supports the two most recent minor ve

Jan 7, 2023
Hierarchical, leveled, and structured logging library for Go

spacelog Please see http://godoc.org/github.com/spacemonkeygo/spacelog for info License Copyright (C) 2014 Space Monkey, Inc. Licensed under the Apach

Apr 27, 2021
Logrus is a structured, pluggable logging for Go.
Logrus is a structured, pluggable logging for Go.

Logrus is a structured logger for Go (golang), completely API compatible with the standard library logger.

May 25, 2021
Minimal structured logging library for Go
Minimal structured logging library for Go

slog slog is a minimal structured logging library for Go. Install go get cdr.dev/slog Features Minimal API First class context.Context support First c

Dec 29, 2022
Fully asynchronous, structured, pluggable logging for Go.

logr Logr is a fully asynchronous, contextual logger for Go. It is very much inspired by Logrus but addresses two issues: Logr is fully asynchronous,

Dec 28, 2022
structured logging helper

Logart Logart is a structured logging tool that aims to simplify logging to a database It is not yet in stable state, but is used in production and ac

Apr 24, 2021
Go-metalog - Standard API for structured logging

Metalog is a standard API for structured logging and adapters for its implementa

Jan 20, 2022
A simple logging module for go, with a rotating file feature and console logging.

A simple logging module for go, with a rotating file feature and console logging. Installation go get github.com/jbrodriguez/mlog Usage Sample usage W

Dec 14, 2022
FactorLog is a logging infrastructure for Go that provides numerous logging functions for whatever your style may be
FactorLog is a logging infrastructure for Go that provides numerous logging functions for whatever your style may be

FactorLog FactorLog is a fast logging infrastructure for Go that provides numerous logging functions for whatever your style may be. It could easily b

Aug 3, 2022
Package logging implements a logging infrastructure for Go
Package logging implements a logging infrastructure for Go

Golang logging library Package logging implements a logging infrastructure for Go. Its output format is customizable and supports different logging ba

Nov 10, 2021
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

The open-source platform for monitoring and observability. Grafana allows you to query, visualize, alert on and understand your metrics no matter wher

Jan 3, 2023
Structured log interface

Structured log interface Package log provides the separation of the logging interface from its implementation and decouples the logger backend from yo

Sep 26, 2022
A minimal and extensible structured logger

⚠️ PRE-RELEASE ⚠️ DO NOT IMPORT THIS MODULE YOUR PROJECT WILL BREAK package log package log provides a minimal interface for structured logging in ser

Jan 7, 2023
Search and analysis tooling for structured logs

Zed The Zed system provides an open-source, cloud-native, and searchable data lake for semi-structured and structured data. Zed lakes utilize a supers

Jan 5, 2023