♻️ The most advanced interruptible mechanism to perform actions repetitively until successful.

♻️ retry Awesome Go

The most advanced interruptible mechanism to perform actions repetitively until successful.

Build Documentation Quality Template Coverage Mirror

💡 Idea

The retry based on Rican7/retry but fully reworked and focused on integration with the 🚧 breaker and the built-in context package.

ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()

action := func(ctx context.Context) (err error) {
	req := req.Clone(ctx)
	resp, err = http.DefaultClient.Do(req)
	return err
}

how := []retry.How{
	strategy.Limit(5),
	strategy.BackoffWithJitter(
		backoff.Fibonacci(10*time.Millisecond),
		jitter.NormalDistribution(
			rand.New(rand.NewSource(time.Now().UnixNano())),
			0.25,
		),
	),
}

if err := retry.Do(ctx, action, how...); err != nil {
	log.Fatal(err)
}

A full description of the idea is available here.

🏆 Motivation

I developed distributed systems at Lazada, and later at Avito, which communicate with each other through a network, and I need a package to make these communications more reliable.

🤼‍♂️ How to

rewriting...

🧩 Integration

The library uses SemVer for versioning, and it is not BC-safe through major releases. You can use go modules to manage its version.

$ go get github.com/kamilsk/retry/v5@latest

🤲 Outcomes

Console tool to execute commands until successful

...

See more details here.

made with ❤️ for everyone

Comments
  • Preserve context values

    Preserve context values

    retry.Do doesn't preserve context values when calling action. Here is an example:

    ctx := context.WithValue(context.Background(), "asdf", 3)
    action := func(ctx context.Context) error {
    	log.Println(ctx.Value("asdf")) // Doesn't print 3
    	return nil
    }
    
    retry.Do(ctx, action)
    

    Does it make sense to change retry.Do to preserve context values?

  • Wrong examples in the README

    Wrong examples in the README

    I'm struggling to follow the examples in the README because I can't relate them to the package actual implementation:

    1. breaker.BreakBySignal as long as the other breakers don't implement the strategy.Breaker interface (the Err() method is missing). This is also a problem in the implementation, that is backing me from using this breaker

    2. The function (second parameter of retry.Do) must accept a context as parameter. This is not reflected in the documentation

  • go get v4 is not working

    go get v4 is not working

    go get -u github.com/kamilsk/retry/v4
    package github.com/kamilsk/retry/v4: cannot find package "github.com/kamilsk/retry/v4" in any of:
    ...
    
  • panic dead lock

    panic dead lock

    If action panic then the done channel is never closed and it hangs forever.

    https://github.com/kamilsk/retry/blob/272c5373f544785efa4a1fd557aadfe505469695/retry.go#L42

  • fix code quality problems

    fix code quality problems

    cmd/retry/main.go:116:14:warning: error return value not checked (spin.Color("red")) (errcheck)
    cmd/retry/main.go:117:37:warning: error return value not checked (color.New(color.FgYellow).Fprintf(stderr, "#%d attempt at %s... \n", attempt+1, time.Now().Sub(start))) (errcheck)
    cmd/retry/main.go:119::warning: Subprocess launching with variable.,HIGH,HIGH (gas)
    cmd/retry/main.go:67:34:warning: error return value not checked (color.New(color.FgRed).Fprintf(app.Stderr, format, err)) (errcheck)
    cmd/retry/main.go:88:39:warning: error return value not checked (color.New(color.FgYellow).Fprintln(stderr, "notify component is not ready yet")) (errcheck)
    cmd/retry/main.go:90:18:warning: error return value not checked (report.Execute(app.Stdout, struct {) (errcheck)
    cmd/retry/parser_test.go:13::warning: cyclomatic complexity 11 of function Test_parse() is high (> 10) (gocyclo)
    
  • add strategy based on classifier package

    add strategy based on classifier package

    the concept:

    func Whitelist(errors ...error) Strategy {
    	whitelist := classifier.WhitelistClassifier(errors)
    	return func(attempt uint, err error) bool {
    		return whitelist.Classify(err) == classifier.Retry
    	}
    }
    
  • add fix util

    add fix util

    $ go get bitbucket.org/kamilsk/retry
    package bitbucket.org/kamilsk/retry: code in directory $GOPATH/src/bitbucket.org/kamilsk/retry expects import "github.com/kamilsk/retry"
    

    blocked by https://github.com/kamilsk/retrier/issues/30

  • unexpected Limit behavior

    unexpected Limit behavior

    origin: https://github.com/Rican7/retry/blob/272ad122d6e5ce1be757544007cf8bcd1c9c9ab0/strategy/strategy.go#L25-L29

    what if:

    var resp *http.Response
    
    action := func(...) error { fill resp }
    
    if err := retry.Do(action, strategy.Limit(0)); err != nil {
        return err
    }
    
    json.NewEncoder(resp.Body).Encode() <- nil pointer, because resp is nil, action didn't run
    
  • goroutine leak in v4

    goroutine leak in v4

    In v4 it is possible to use "infinite" context to call inside an action

    	err = retry.Try(
    		breaker.BreakByTimeout(10*time.Second),
    		func(uint) error {
    			statuses, err = s.dwhClient.V1Status(ctx, source)
    			return err
    		},
    		strategy.Limit(10),
    		strategy.Wait(time.Millisecond*10, time.Millisecond*100, time.Millisecond*1000),
    	)
    	return
    

    The problem is that if the client doesn't have read/conn timeout the V1Status can block goroutine for a long time.

  • extend breaker interface

    extend breaker interface

    // A Breaker carries a cancellation signal to break an action execution.
    //
    // It is a subset of context.Context and github.com/kamilsk/breaker.Breaker.
    type Breaker interface {
    	// Done returns a channel that's closed when a cancellation signal occurred.
    	Done() <-chan struct{}
    
    	Error() error
    }
    
  • TryContext doesn't work properly

    TryContext doesn't work properly

    The main problem is that ctx is not interrupted by func retry.

    Possible solution without breaking changes:

    func TryContext(
    	ctx context.Context,
    	action func(ctx context.Context, attempt uint) error,
    	strategies ...func(attempt uint, err error) bool,
    ) error {
    	ctx, cancel := context.WithCancel(ctx)
    	err := retry(ctx, currying(ctx, action), strategies...)
    	cancel() // or defer it
    	return err
    }
    

    Also, I have to delete

    	if (breaker == nil || breaker.Done() == nil) && len(strategies) == 0 {
    		return action(0)
    	}
    
  • prepare to switch to go.octolab.org/retry

    prepare to switch to go.octolab.org/retry

    1. move the repo to octolab org
    2. fork the repo
    3. use https://tip.golang.org/ref/mod#go-mod-file-module-deprecation
    4. write documentation
    5. update awesome list
    6. archive the repo
  • mark CheckError as experimental feature and add typical mistakes

    mark CheckError as experimental feature and add typical mistakes

    https://github.com/kamilsk/retry/blob/19175c70a27be2f7fda74b27ec2b8551fec29e5f/strategy/strategy.go#L102-L144

    TODO:

    • [ ] find bad case usage from goreleaser
  • update consumers

    update consumers

    • https://pkg.go.dev/github.com/kamilsk/retry/v4?tab=importedby
      • [ ] https://github.com/tmaiaroto/aegis, v4.1.0 -> v5.0.0
      • [x] ~https://github.com/goreleaser/goreleaser, v4.7.2 -> v5.0.0~
Go-github-actions - `go-github-actions` is a package for developing GitHub Actions

go-github-actions go-github-actions is a package for developing GitHub Actions.

Feb 6, 2022
keeper is package for Go that provides a mechanism for waiting a result of execution function until context cancel.

keeper is package for Go that provides a mechanism for waiting a result of execution function until context cancel.

Apr 18, 2022
⚗ The most advanced CLI template on earth! Featuring automatic releases, website generation and a custom CI-System out of the box.
⚗ The most advanced CLI template on earth! Featuring automatic releases, website generation and a custom CI-System out of the box.

cli-template ✨ ⚗ A template for beautiful, modern, cross-platform compatible CLI tools written with Go! Getting Started | Wiki This template features

Dec 4, 2022
github-actions-merger is github actions that merges pull request with commit message including pull request labels.

github-actions-merger github-actions-merger is github actions that merges pull request with commit message including pull request labels. Usage Write

Dec 7, 2022
This map is totally blank for me. Let's discover it until we find the treasure!

Go Playground Yet another playground project. This time to learn a bit of Go! Know the business details about this project looking at the docs folder.

Sep 15, 2021
Birthdays is a web service that stores the birthday date of users and calculates the remaining days until the user's birthday.
Birthdays is a web service that stores the birthday date of users and calculates the remaining days until the user's birthday.

Birthdays is a web service that stores the birthday date of users and calculates the remaining days until the user's birthday. Features Metrics servic

May 2, 2022
Proxy that keeps clients active until the backend server is back online

HoneySmoke HoneySmoke is a prototype proxy for testing until it eventually becomes HoneyHive. HoneySmoke will eventually implement a limbo mode that k

Nov 20, 2021
Feb 12, 2022
A wrapper for leader election mechanism in Kubernetes

Kubernetes Leader Election Library for Go This library provides a thin wrapper for kubernetes leader election. It can be used to elect a leader betwee

Dec 24, 2022
Deece is an open, collaborative, and decentralised search mechanism for IPFS
Deece is an open, collaborative, and decentralised search mechanism for IPFS

Deece Deece is an open, collaborative, and decentralised search mechanism for IPFS. Any node running the client is able to crawl content on IPFS and a

Oct 29, 2022
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).

goInterLock is golang job/task scheduler with distributed locking mechanism. In distributed system locking is preventing task been executed in every instant that has the scheduler,

Dec 5, 2022
Declarative CLI Version manager. Support Lazy Install and Sharable configuration mechanism named Registry. Switch versions seamlessly

aqua Declarative CLI Version manager. Support Lazy Install and Sharable configuration mechanism named Registry. Switch versions seamlessly. Index Slid

Dec 29, 2022
A universal mechanism to manage goroutine lifecycles

A universal mechanism to manage goroutine lifecycles

Dec 29, 2022
Multithreaded key value pair store using thread safe locking mechanism allowing concurrent reads
Multithreaded key value pair store using thread safe locking mechanism allowing concurrent reads

Project Amnesia A Multi-threaded key-value pair store using thread safe locking mechanism allowing concurrent reads. Curious to Try it out?? Check out

Oct 29, 2022
The Hyperscale InputFilter library provides a simple inputfilter chaining mechanism by which multiple filters and validator may be applied to a single datum in a user-defined order.

Hyperscale InputFilter Branch Status Coverage master The Hyperscale InputFilter library provides a simple inputfilter chaining mechanism by which mult

Oct 20, 2021
Study project that uses Apache Kafka as syncing mechanism between two databases, with producers and consumers written in Go.

Kafka DB Sync Study project that uses Apache Kafka as syncing mechanisms between a monolith DB and a microservice. The main purpose of this project is

Dec 5, 2021
One Time Passwords (OTPs) are an mechanism to improve security over passwords alone.

otp: One Time Password utilities Go / Golang Why One Time Passwords? One Time Passwords (OTPs) are an mechanism to improve security over passwords alo

Jan 7, 2023
Goya circuit is a circuit breaker mechanism implementation in microservices.

Goya-Circuit: 类似于Hystrix的熔断器实现 Goya circuit is a circuit breaker mechanism implementation in microservices. It can prevent the whole link avalanche ca

Mar 8, 2022
Automatically capture all potentially useful information about each executed command (as well as its output) and get powerful querying mechanism
Automatically capture all potentially useful information about each executed command (as well as its output) and get powerful querying mechanism

nhi is a revolutionary tool which automatically captures all potentially useful information about each executed command and everything around, and delivers powerful querying mechanism.

Nov 29, 2022
Utility library that uses Go generics mechanism

golang-generics-util Utility library that explores Go generics (1.18) xsync Sync

Dec 11, 2022