Fast strftime for Go

strftime

Fast strftime for Go

Build Status

GoDoc

SYNOPSIS

f := strftime.New(`.... pattern ...`)
if err := f.Format(buf, time.Now()); err != nil {
    log.Println(err.Error())
}

DESCRIPTION

The goals for this library are

  • Optimized for the same pattern being called repeatedly
  • Be flexible about destination to write the results out
  • Be as complete as possible in terms of conversion specifications

API

Format(string, time.Time) (string, error)

Takes the pattern and the time, and formats it. This function is a utility function that recompiles the pattern every time the function is called. If you know beforehand that you will be formatting the same pattern multiple times, consider using New to create a Strftime object and reuse it.

New(string) (*Strftime, error)

Takes the pattern and creates a new Strftime object.

obj.Pattern() string

Returns the pattern string used to create this Strftime object

obj.Format(io.Writer, time.Time) error

Formats the time according to the pre-compiled pattern, and writes the result to the specified io.Writer

obj.FormatString(time.Time) string

Formats the time according to the pre-compiled pattern, and returns the result string.

SUPPORTED CONVERSION SPECIFICATIONS

pattern description
%A national representation of the full weekday name
%a national representation of the abbreviated weekday
%B national representation of the full month name
%b national representation of the abbreviated month name
%C (year / 100) as decimal number; single digits are preceded by a zero
%c national representation of time and date
%D equivalent to %m/%d/%y
%d day of the month as a decimal number (01-31)
%e the day of the month as a decimal number (1-31); single digits are preceded by a blank
%F equivalent to %Y-%m-%d
%H the hour (24-hour clock) as a decimal number (00-23)
%h same as %b
%I the hour (12-hour clock) as a decimal number (01-12)
%j the day of the year as a decimal number (001-366)
%k the hour (24-hour clock) as a decimal number (0-23); single digits are preceded by a blank
%l the hour (12-hour clock) as a decimal number (1-12); single digits are preceded by a blank
%M the minute as a decimal number (00-59)
%m the month as a decimal number (01-12)
%n a newline
%p national representation of either "ante meridiem" (a.m.) or "post meridiem" (p.m.) as appropriate.
%R equivalent to %H:%M
%r equivalent to %I:%M:%S %p
%S the second as a decimal number (00-60)
%T equivalent to %H:%M:%S
%t a tab
%U the week number of the year (Sunday as the first day of the week) as a decimal number (00-53)
%u the weekday (Monday as the first day of the week) as a decimal number (1-7)
%V the week number of the year (Monday as the first day of the week) as a decimal number (01-53)
%v equivalent to %e-%b-%Y
%W the week number of the year (Monday as the first day of the week) as a decimal number (00-53)
%w the weekday (Sunday as the first day of the week) as a decimal number (0-6)
%X national representation of the time
%x national representation of the date
%Y the year with century as a decimal number
%y the year without century as a decimal number (00-99)
%Z the time zone name
%z the time zone offset from UTC
%% a '%'

EXTENSIONS / CUSTOM SPECIFICATIONS

This library in general tries to be POSIX compliant, but sometimes you just need that extra specification or two that is relatively widely used but is not included in the POSIX specification.

For example, POSIX does not specify how to print out milliseconds, but popular implementations allow %f or %L to achieve this.

For those instances, strftime.Strftime can be configured to use a custom set of specifications:

ss := strftime.NewSpecificationSet()
ss.Set('L', ...) // provide implementation for `%L`

// pass this new specification set to the strftime instance
p, err := strftime.New(`%L`, strftime.WithSpecificationSet(ss))
p.Format(..., time.Now())

The implementation must implement the Appender interface, which is

type Appender interface {
  Append([]byte, time.Time) []byte
}

For commonly used extensions such as the millisecond example and Unix timestamp, we provide a default implementation so the user can do one of the following:

// (1) Pass a specification byte and the Appender
//     This allows you to pass arbitrary Appenders
p, err := strftime.New(
  `%L`,
  strftime.WithSpecification('L', strftime.Milliseconds),
)

// (2) Pass an option that knows to use strftime.Milliseconds
p, err := strftime.New(
  `%L`,
  strftime.WithMilliseconds('L'),
)

Similarly for Unix Timestamp:

// (1) Pass a specification byte and the Appender
//     This allows you to pass arbitrary Appenders
p, err := strftime.New(
  `%s`,
  strftime.WithSpecification('s', strftime.UnixSeconds),
)

// (2) Pass an option that knows to use strftime.UnixSeconds
p, err := strftime.New(
  `%s`,
  strftime.WithUnixSeconds('s'),
)

If a common specification is missing, please feel free to submit a PR (but please be sure to be able to defend how "common" it is)

List of available extensions

PERFORMANCE / OTHER LIBRARIES

The following benchmarks were run separately because some libraries were using cgo on specific platforms (notabley, the fastly version)

// On my OS X 10.14.6, 2.3 GHz Intel Core i5, 16GB memory.
// go version go1.13.4 darwin/amd64
hummingbird% go test -tags bench -benchmem -bench .
<snip>
BenchmarkTebeka-4                 	  297471	      3905 ns/op	     257 B/op	      20 allocs/op
BenchmarkJehiah-4                 	  818444	      1773 ns/op	     256 B/op	      17 allocs/op
BenchmarkFastly-4                 	 2330794	       550 ns/op	      80 B/op	       5 allocs/op
BenchmarkLestrrat-4               	  916365	      1458 ns/op	      80 B/op	       2 allocs/op
BenchmarkLestrratCachedString-4   	 2527428	       546 ns/op	     128 B/op	       2 allocs/op
BenchmarkLestrratCachedWriter-4   	  537422	      2155 ns/op	     192 B/op	       3 allocs/op
PASS
ok  	github.com/lestrrat-go/strftime	25.618s
// On a host on Google Cloud Platform, machine-type: f1-micro (vCPU x 1, memory: 0.6GB)
// (Yes, I was being skimpy)
// Linux <snip> 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20) x86_64 GNU/Linux
// go version go1.13.4 linux/amd64
hummingbird% go test -tags bench -benchmem -bench .
<snip>
BenchmarkTebeka                   254997              4726 ns/op             256 B/op         20 allocs/op
BenchmarkJehiah                   659289              1882 ns/op             256 B/op         17 allocs/op
BenchmarkFastly                   389150              3044 ns/op             224 B/op         13 allocs/op
BenchmarkLestrrat                 699069              1780 ns/op              80 B/op          2 allocs/op
BenchmarkLestrratCachedString    2081594               589 ns/op             128 B/op          2 allocs/op
BenchmarkLestrratCachedWriter     825763              1480 ns/op             192 B/op          3 allocs/op
PASS
ok      github.com/lestrrat-go/strftime 11.355s

This library is much faster than other libraries IF you can reuse the format pattern.

Here's the annotated list from the benchmark results. You can clearly see that (re)using a Strftime object and producing a string is the fastest. Writing to an io.Writer seems a bit sluggish, but since the one producing the string is doing almost exactly the same thing, we believe this is purely the overhead of writing to an io.Writer

Import Path Score Note
github.com/lestrrat-go/strftime 3000000 Using FormatString() (cached)
github.com/fastly/go-utils/strftime 2000000 Pure go version on OS X
github.com/lestrrat-go/strftime 1000000 Using Format() (NOT cached)
github.com/jehiah/go-strftime 1000000
github.com/fastly/go-utils/strftime 1000000 cgo version on Linux
github.com/lestrrat-go/strftime 500000 Using Format() (cached)
github.com/tebeka/strftime 300000

However, depending on your pattern, this speed may vary. If you find a particular pattern that seems sluggish, please send in patches or tests.

Please also note that this benchmark only uses the subset of conversion specifications that are supported by ALL of the libraries compared.

Somethings to consider when making performance comparisons in the future:

  • Can it write to io.Writer?
  • Which %specification does it handle?
Owner
@lestrrat 's Go projects
null
Comments
  • replace deprecated package github.com/pkg/errors

    replace deprecated package github.com/pkg/errors

    Replaces github.com/pkg/errors, which is now marked as archived and no longer maintained on github, with native >=go1.13 standard library functions. This preserves the error wrapping previously handled via errors.Wrap via the new %w directive in fmt.Errorf.

    This makes the minimum go version go1.13, so thus update the go.mod to reflect and also updates the test matrix here to test from go1.13-go1.18 inclusive (previously tested go1.14-go1.15).

  • Fix issue with optimization on verbatim value

    Fix issue with optimization on verbatim value

    Apparently our tests did not include a case where there were multiple verbatim sections in a pattern, which made us oblivious to this issue. Moving the verbatim value inside the loop does the trick.

  • Configurable Specifications

    Configurable Specifications

    In #7 we got a request to add %L as a specification to generate textual representation of milliseconds from the given time. This in itself is totally fine, except %L is not in POSIX, and implementations differ on the exact pattern to invoke this (e.g. %f or %L).

    While #7 just decided that %L was the right choice, I'd like to take this opportunity and introduce enough flexibility that we can create different flavors of Strftime in the future when the needs arises.

    This PR takes the previously static list of specifications, and moves them in a map so that they are more easily configurable. For common specifications, an optional parameter in the various methods can dynamically alter the specification set so that one instance of Strftime can generate a completely different set of text representations if need be.

    If in the future we have enough flavors to work with, such as "the pythin strftime" or "the ruby strftime", we could just wrap the objects as:

    ... = strftime.Python.Format(t)
    ... = strftime.Ruby.Format(t)
    ... = strftime.POSIX.Format(t) // current default
    

    And users can more easily navigate through the different implementations.

  • It is not same with c strftime?

    It is not same with c strftime?

    for i := 0; i < 400; i++ {
    		now := time.Now().Unix() + int64(i)*86400
    		cWeek := common.GetWeek(now) // cgo
    
    		strfWeek, _ := strftime.Format("%Y%U", time.Unix(now, 0))
    		gWeek, _ := strconv.ParseInt(strfWeek, 10, 64)
    		if cWeek != int(gWeek) {
    			fmt.Println("failed!!!", time.Unix(now, 0).Format("2006.01.02 15:04:05"), cWeek, gWeek)
    			break
    		}
    	}
    

    output: failed!!! 2021.01.01 00:35:27 202052 202100 failed!!! 2021.01.02 00:35:27 202052 202100

  • Add `%L` pattern to support millisecond [000..999]

    Add `%L` pattern to support millisecond [000..999]

    But I'm not confident whether %L is a suitable to do it because that one is not supported by POSIX strftime(3). AFAIK, ruby provides %L and python provides %f for that.

  • build(ci): test matrix include go1.12 up to go1.18

    build(ci): test matrix include go1.12 up to go1.18

    This tests back from the version specified in go.mod to current stable.

    This should enable detection of any breaking version changes as part of the dependencies update PR.

    subtask forked from #28.

  • If you have a number before a specification, it breaks the compiled pattern

    If you have a number before a specification, it breaks the compiled pattern

    Whilst using this library, I found an interesting bug(?) issue(?) to create log file names.

    If I use: pattern, _ := strftime.New("/path/test%Y%m%d.log") I get pattern -> "/path/test20191230.log" Awesome.

    But. If I use: pattern, _ := strftime.New("/path/test1%Y%m%d.log") I get pattern -> ".log20191230.log" Not awesome.

    If I try: pattern, _ := strftime.New("/longer/path/to/see/where/this/leads//test99%Y%m%d.log") I get pattern -> ".log20191230.log" Also not awesome.

    This has only started post PR#6 as far as I can tell, as I have an older forked version that does not have this issue.

    Am I going mad?

  • Add FormatBuffer

    Add FormatBuffer

    As discussed in issue 25, this is structurally similar to the standard library's AppendFormat method. I also fixed some issues with BenchmarkLestrratCachedWriter which made it look slower than it really was.

    Nets about a 15% speedup which is less than I was hoping for, but still worth using in performance-intensive situations. Note the difference in the time output though - FormatBuffer uses about 20% less CPU time in total for the same number of executions, probably due to reduced garbage collection overhead.

    [master][~/hound/go/src/github.com/ianwilkes/strftime/bench]$ time gotest . -bench BenchmarkLestrratCachedWriter -benchtime 30000000x
    goos: darwin
    goarch: amd64
    pkg: github.com/lestrrat-go/strftime/bench
    cpu: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    BenchmarkLestrratCachedWriter-8     30000000         390.1 ns/op
    PASS
    ok    github.com/lestrrat-go/strftime/bench 11.856s
    gotest . -bench BenchmarkLestrratCachedWriter -benchtime 30000000x  12.69s user 1.07s system 111% cpu 12.299 total
    
    [master][~/hound/go/src/github.com/ianwilkes/strftime/bench]$ time gotest . -bench BenchmarkLestrratCachedFormatBuffer -benchtime 30000000x
    goos: darwin
    goarch: amd64
    pkg: github.com/lestrrat-go/strftime/bench
    cpu: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
    BenchmarkLestrratCachedFormatBuffer-8     30000000         330.3 ns/op
    PASS
    ok    github.com/lestrrat-go/strftime/bench 10.063s
    gotest . -bench BenchmarkLestrratCachedFormatBuffer -benchtime 30000000x  10.49s user 0.62s system 105% cpu 10.517 total
    
  • Allocation in every Format() call

    Allocation in every Format() call

    This is purely a performance issue, but a significant one. This library could be much faster if it could be used in a zero-allocation way, writing into a buffer provided by the caller. Today, this array leaks onto the heap because it's passed into Write().

    A simple fix would be to keep that array as a field of the Strftime struct, but then this method would be concurrency unsafe. (The io.Writer spec doesn't allow implementations to modify or retain references to the passed slice, so you can get away with re-using that buffer, but obviously not with concurrent use.)

    I think a better approach would be an exported version of the internal format method - maybe just implement Append on Strftime. That way, the user can supply the buffer, and allocation only occurs if it's insufficient.

    I'm happy to submit a PR if the approach sounds good.

  • Promote %s to a standard specification?

    Promote %s to a standard specification?

    I was writing docs on what strftime specs are supported for a project of mine using this library, so I went to look for where the specs originated. Turns out they're from FreeBSD[1]/macOS[2] strftime(3): the descriptions match word for word, and the lists of specs are the same with a few specs/categories missing here — POSIX locale extensions (%E*, %O*), glibc extensions, %G, %g and %s. I can understand not implementing %G and %g, they're weird (%G for 2019-12-31 => 2020), but %s is trivial and very widely supported and used, so it's a bit strange it has to be added as an extension, seems more like an accidental omission to me.

    So what about promoting it to a standard spec? (The extension could of course be kept around for backward compatibility.) It's an easy change, I can submit a PR if you don't feel like spending time on this.

    [1] https://www.freebsd.org/cgi/man.cgi?query=strftime&sektion=3 [2] https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/strftime.3.html

  • add go.sum via go mod tidy

    add go.sum via go mod tidy

    go.sum should be checked into VCS as per official documentation at: https://github.com/golang/go/wiki/Modules#releasing-modules-all-versions https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file

    subtask forked from #28.

  • Hyphenated formats for non-zero padded numbers

    Hyphenated formats for non-zero padded numbers

    Some strftime implementations support non-zero padded numbers.

    For example, in Python

    >>> from datetime import datetime
    >>> datetime.strftime(datetime.now(), '%-d')
    '1'
    

    This is documented in glibc's strftime.

    Windows has its own set of codes which use # instead of -: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/strftime-wcsftime-strftime-l-wcsftime-l?redirectedfrom=MSDN&view=vs-2019

    Another reference: https://strftime.org/#platforms

strftime for Go

gostrftime About A Go pkg for formatting time.Time in an strftime(3) like way. Basically, strftime for Go, with a couple of additions. Installing $ go

Oct 21, 2021
A fast ISO8601 date parser for Go

A fast ISO8601 date parser for Go go get github.com/relvacode/iso8601 The built-in RFC3333 time layout in Go is too restrictive to support any ISO860

Jan 7, 2023
fasttime - fast time formatting for go

fasttime - fast time formatting for go

Dec 13, 2022
timeutil - useful extensions (Timedelta, Strftime, ...) to the golang's time package

timeutil - useful extensions to the golang's time package timeutil provides useful extensions (Timedelta, Strftime, ...) to the golang's time package.

Dec 22, 2022
strftime for Go

gostrftime About A Go pkg for formatting time.Time in an strftime(3) like way. Basically, strftime for Go, with a couple of additions. Installing $ go

Oct 21, 2021
llb - It's a very simple but quick backend for proxy servers. Can be useful for fast redirection to predefined domain with zero memory allocation and fast response.

llb What the f--k it is? It's a very simple but quick backend for proxy servers. You can setup redirect to your main domain or just show HTTP/1.1 404

Sep 27, 2022
Chisel is a fast TCP/UDP tunnel, transported over HTTP, secured via SSH.
Chisel is a fast TCP/UDP tunnel, transported over HTTP, secured via SSH.

Chisel is a fast TCP/UDP tunnel, transported over HTTP, secured via SSH. Single executable including both client and server. Written in Go (golang). Chisel is mainly useful for passing through firewalls, though it can also be used to provide a secure endpoint into your network.

Jan 1, 2023
Safe, simple and fast JSON Web Tokens for Go

jwt JSON Web Token for Go RFC 7519, also see jwt.io for more. The latest version is v3. Rationale There are many JWT libraries, but many of them are h

Jan 4, 2023
Fast, secure and efficient secure cookie encoder/decoder

Encode and Decode secure cookies This package provides functions to encode and decode secure cookie values. A secure cookie has its value ciphered and

Dec 9, 2022
A simple, fast, and fun package for building command line apps in Go

cli cli is a simple, fast, and fun package for building command line apps in Go. The goal is to enable developers to write fast and distributable comm

Dec 31, 2022
Fast, secure, efficient backup program
Fast, secure, efficient backup program

Introduction restic is a backup program that is fast, efficient and secure. It supports the three major operating systems (Linux, macOS, Windows) and

Dec 31, 2022
A fast and powerful alternative to grep

sift A fast and powerful open source alternative to grep. Features sift has a slightly different focus than most other grep alternatives. Code search,

Jan 3, 2023
Experiment - Sync files to S3, fast. Go package and CLI.

gosync I want to be the fastest way to concurrently sync files and directories to/from S3. Gosync will concurrently transfer your files to and from S3

Nov 3, 2022
Alternative archiving tool with fast performance for huge numbers of small files

fast-archiver fast-archiver is a command-line tool for archiving directories, and restoring those archives written in [Go](http://golang.org). fast-ar

Nov 22, 2022
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.

Walrus is a fast, secure and reliable backup system suitable for modern infrastructure. With walrus, you can backup services like MySQL, PostgreSQL, Redis, etcd or a complete directory with a short interval and low overhead. It supports AWS S3, digitalocean spaces and any S3-compatible object storage service.

Jan 5, 2023
Go implementation of SipHash-2-4, a fast short-input PRF created by Jean-Philippe Aumasson and Daniel J. Bernstein.

SipHash (Go) Go implementation of SipHash-2-4, a fast short-input PRF created by Jean-Philippe Aumasson and Daniel J. Bernstein (http://131002.net/sip

Dec 25, 2022
Fast ring-buffer deque (double-ended queue)

deque Fast ring-buffer deque (double-ended queue) implementation. For a pictorial description, see the Deque diagram Installation $ go get github.com/

Dec 26, 2022
Go native library for fast point tracking and K-Nearest queries

Geo Index Geo Index library Overview Splits the earth surface in a grid. At each cell we can store data, such as list of points, count of points, etc.

Dec 3, 2022
Fast in-memory key:value store/cache with TTL

MCache library go-mcache - this is a fast key:value storage. Its major advantage is that, being essentially a thread-safe . map[string]interface{} wit

Nov 11, 2022
Data structure and relevant algorithms for extremely fast prefix/fuzzy string searching.

Trie Data structure and relevant algorithms for extremely fast prefix/fuzzy string searching. Usage Create a Trie with: t := trie.New() Add Keys with:

Dec 27, 2022