Package tasks is an easy to use in-process scheduler for recurring tasks in Go

Tasks

Build Status Coverage Status Go Report Card Documentation

Package tasks is an easy to use in-process scheduler for recurring tasks in Go. Tasks is focused on high frequency tasks that run quick, and often. The goal of Tasks is to support concurrent running tasks at scale without scheduler induced jitter.

Tasks is focused on accuracy of task execution. To do this each task is called within it's own goroutine. This ensures that long execution of a single invocation does not throw the schedule as a whole off track.

As usage of this scheduler scales, it is expected to have a larger number of sleeping goroutines. As it is designed to leverage Go's ability to optimize goroutine CPU scheduling.

For simplicity this task scheduler uses the time.Duration type to specify intervals. This allows for a simple interface and flexible control over when tasks are executed.

Below is an example of starting the scheduler and registering a new task that runs every 30 seconds.

// Start the Scheduler
scheduler := tasks.New()
defer scheduler.Stop()

// Add a task
id, err := scheduler.Add(&tasks.Task{
  Interval: time.Duration(30 * time.Second),
  TaskFunc: func() error {
    // Put your logic here
  }(),
})
if err != nil {
  // Do Stuff
}

Sometimes schedules need to started at a later time. This package provides the ability to start a task only after a certain time. The below example shows this in practice.

// Add a recurring task for every 30 days, starting 30 days from now
id, err := scheduler.Add(&tasks.Task{
  Interval: time.Duration(30 * (24 * time.Hour)),
  StartAfter: time.Now().Add(30 * (24 * time.Hour)),
  TaskFunc: func() error {
    // Put your logic here
  }(),
})
if err != nil {
  // Do Stuff
}

It is also common for applications to run a task only once. The below example shows scheduling a task to run only once after waiting for 60 seconds.

// Add a one time only task for 60 seconds from now
id, err := scheduler.Add(&tasks.Task{
  Interval: time.Duration(60 * time.Second)
  RunOnce:  true,
  TaskFunc: func() error {
    // Put your logic here
  }(),
})
if err != nil {
  // Do Stuff
}

One powerful feature of Tasks is that it allows users to specify custom error handling. This is done by allowing users to define a function that is called when a task returns an error. The below example shows scheduling a task that logs when an error occurs.

// Add a task with custom error handling
id, err := scheduler.Add(&tasks.Task{
  Interval: time.Duration(30 * time.Second),
  TaskFunc: func() error {
    // Put your logic here
  }(),
  ErrFunc: func(e error) {
    log.Printf("An error occurred when executing task %s - %s", id, e)
  }(),
})
if err != nil {
  // Do Stuff
}
Owner
Benjamin Cane
Thoughts and opinions are my own.
Benjamin Cane
Comments
  • fix the warnings for go test --race

    fix the warnings for go test --race

    Description

    As title.

    Type of change

    • [x] Bug fix (non-breaking change which fixes an issue)

    Checklist:

    • [x] I have performed a self-review of my own code
    • [x] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [ ] I have made corresponding changes to documentation
    • [ ] I have added tests that ensure my fix is effective or that my feature works
    • [ ] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Added AddWithID function

    Added AddWithID function

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    Fixes # (issue)

    Type of change

    Please delete options that are not relevant.

    • [x] New feature (non-breaking change which adds functionality)

    Checklist:

    • [x] I have performed a self-review of my own code
    • [x] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [x] I have made corresponding changes to documentation
    • [x] I have added tests that ensure my fix is effective or that my feature works
    • [x] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Copying Map

    Copying Map

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    This change will create a copy of the task map and return it to users rather than returning the task map directly. Since maps in Go are essentially pointers, if a user is iterating over the returned map while it's being written to (scheduling, deleting, etc.) then there is a possibility of a race condition.

    This change will prevent that by returning a copy of the map.

    Fixes # (issue)

    Type of change

    Please delete options that are not relevant.

    • [X] Bug fix (non-breaking change which fixes an issue)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Adding delay only for late starters

    Adding delay only for late starters

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    This PR fixes an interesting issue reported by users where quick scheduling and removal can cause timers to remain. This moves the schedule into the hot path only if StartAfter is 0.

    Fixes # (issue)

    Type of change

    Please delete options that are not relevant.

    • [X] Bug fix (non-breaking change which fixes an issue)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • A few optimizations

    A few optimizations

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    Fixes # (issue)

    Cleaning up a bit of duplicate code and optimizing just a little bit.

    Type of change

    Please delete options that are not relevant.

    • [X] Small and simple changes (Tech Debt, Doc updates, Adding tests, etc.)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • GitHub Actions and Benchmarks

    GitHub Actions and Benchmarks

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    Fixes # (issue)

    This pull request has a few changes in it.

    • Removes vendor directory (no longer needed)
    • Introduces GH Actions for linting and executing tests
    • Introduces some simple benchmarks

    Type of change

    Please delete options that are not relevant.

    • [X] New feature (non-breaking change which adds functionality)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Fixing silly cyclo rules

    Fixing silly cyclo rules

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    Fixes comments from: https://github.com/avelino/awesome-go/pull/3497

    This PR fixes GoCyclo scores which was mainly inflated due to using a bunch of sub-tests under 1 Test.

    Type of change

    Please delete options that are not relevant.

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [ ] This change requires a documentation update
    • [X] Small and simple changes (Tech Debt, Doc updates, Adding tests, etc.)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

  • Moving to AfterFunc which should be more efficient

    Moving to AfterFunc which should be more efficient

    Description

    Moving from using Ticker and a few goroutines to a Timer using the AfterFunc call. This should in theory be much more efficient since it's managed by the runtime and doesn't require a waiting goroutine.

    Type of change

    Please delete options that are not relevant.

    • [X] Small and simple changes (Tech Debt, Doc updates, Adding tests, etc.)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [ ] I have added tests that ensure my fix is effective or that my feature works
    • [X] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

    New tests are not required for this, more important that existing tests pass.

  • Start scheduler after X moment in time

    Start scheduler after X moment in time

    Description

    This pull request adds the ability to tell the scheduler not to schedule a task until after X point in time. This is useful for when you want to schedule something to start in the future but not now. I.E. I want to start an email task every Sunday, I could start the scheduler by waiting X days until next Sunday and give the task an interval of 7 days.

    Type of change

    • [X] New feature (non-breaking change which adds functionality)
    • [X] Small and simple changes (Tech Debt, Doc updates, Adding tests, etc.)

    Checklist:

    • [X] I have performed a self-review of my own code
    • [X] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [X] I have made corresponding changes to documentation
    • [X] I have added tests that ensure my fix is effective or that my feature works
    • [ ] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

    No downstream dependencies.

  • [feat] task err func: add task's id access

    [feat] task err func: add task's id access

    Description

    Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

    Fixes # (issue)

    Type of change

    Please delete options that are not relevant.

    • [ ] Bug fix (non-breaking change which fixes an issue)
    • [ ] New feature (non-breaking change which adds functionality)
    • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • [ ] This change requires a documentation update
    • [ ] Small and simple changes (Tech Debt, Doc updates, Adding tests, etc.)

    Checklist:

    • [ ] I have performed a self-review of my own code
    • [ ] I have commented my code, ensuring GoDoc readability and clarity in hard-to-understand areas
    • [ ] I have made corresponding changes to documentation
    • [ ] I have added tests that ensure my fix is effective or that my feature works
    • [ ] Any dependent changes have been merged and published in downstream modules

    If checklist items are unchecked please explain.

nano-gpu-scheduler is a Kubernetes scheduler extender for GPU resources scheduling.
nano-gpu-scheduler is a Kubernetes scheduler extender for GPU resources scheduling.

Nano GPU Scheduler About This Project With the continuous evolution of cloud native AI scenarios, more and more users run AI tasks on Kubernetes, whic

Dec 29, 2022
Statefulset-scheduler (aka sfs-scheduler)

statefulset-scheduler (aka sfs-scheduler) Installation I already upload docker i

Dec 19, 2021
Scheduler: the scheduler of distbuild written in Golang

scheduler Introduction scheduler is the scheduler of distbuild written in Go. Pr

Feb 9, 2022
Linstor-scheduler-extender - LINSTOR scheduler extender plugin for Kubernetes

linstor-scheduler-extender LINSTOR scheduler extender plugin for Kubernetes whic

Dec 30, 2022
Crane scheduler is a Kubernetes scheduler which can schedule pod based on actual node load.

Crane-scheduler Overview Crane-scheduler is a collection of scheduler plugins based on scheduler framework, including: Dynamic scheuler: a load-aware

Dec 29, 2022
A lightweight job scheduler based on priority queue with timeout, retry, replica, context cancellation and easy semantics for job chaining. Build for golang web apps.

Table of Contents Introduction What is RIO? Concern An asynchronous job processor Easy management of these goroutines and chaining them Introduction W

Dec 9, 2022
Delay-tasks - A delayed tasks implementation for golang
Delay-tasks - A delayed tasks implementation for golang

delay-tasks An implementation of delayed tasks. Usage $ git clone https://github

Jan 14, 2022
GPU Sharing Scheduler for Kubernetes Cluster
GPU Sharing Scheduler for Kubernetes Cluster

GPU Sharing Scheduler Extender in Kubernetes Overview More and more data scientists run their Nvidia GPU based inference tasks on Kubernetes. Some of

Jan 6, 2023
A simple job scheduler backed by Postgres.

A simple job scheduler backed by Postgres used in production at https://operand.ai. Setup needs two environment variables, SECRET and ENDPOINT. The se

Sep 10, 2022
Chrono is a scheduler library that lets you run your task and code periodically
Chrono is a scheduler library that lets you run your task and code periodically

Chrono is a scheduler library that lets you run your tasks and code periodically. It provides different scheduling functionalities to make it easier t

Dec 26, 2022
cpuworker - A Customized Goroutine Scheduler over Golang Runtime
cpuworker - A Customized Goroutine Scheduler over Golang Runtime

cpuworker Status Working in process. Run the Demo Make sure the GOMAXPROCS is bigger than 1 and there is at least GOMAXPROCS physical OS threads avail

Dec 6, 2022
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).

goInterLock is golang job/task scheduler with distributed locking mechanism. In distributed system locking is preventing task been executed in every instant that has the scheduler,

Dec 5, 2022
Chadburn is a scheduler alternative to cron, built on Go and designed for Docker environments.

Chadburn - a job scheduler Chadburn is a modern and low footprint job scheduler for docker environments, written in Go. Chadburn aims to be a replacem

Dec 6, 2022
A Framework for FaaS load balancing | stack-scheduler repository|

P2PFaaS A Framework for FaaS load balancing | stack-scheduler repository Introduction The P2PFaaS is a framework that allows you to implement a load b

Oct 29, 2021
Go distributed task scheduler

Go distributed task scheduler

Nov 13, 2021
A sample to showcase how to create a k8s scheduler extender

sample-scheduler-extender A sample to showcase how to create a k8s scheduler extender. UPDATE on 2020.6.10 Switch go module, and wire dependencies to

Nov 17, 2021
Scheduler CRUD For Golang
Scheduler CRUD For Golang

scheduler-CRUD 從dbdiagram.io建立table與create語法 在mysql建立table 以sqlc建CRUD function與Transaction 加入viper讀環境變量 以gin開發REST API 加入Dockerfile docker build -t th

Feb 10, 2022
Scheduler: Go jobs execution system

Scheduler Go jobs execution system. Inspired by CI/CD and Unity task scheduler.

Jul 1, 2022
personal tweet scheduler - it needs my guidance now for it to work for you - it works on my mac - will release it someday

tit tit daemon write tests automate build & install plist replace {{path_for_titd_executable}} accordingly. <?xml version="1.0" encoding="UTF-8"?> <!D

Feb 4, 2022