🕰 Sched

Go In-Process Scheduler with Cron Expression Support

Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

Introduction

A simple process manager that allows you to specify a Schedule that execute a Job based on a Timer. Schedule manage the state of this job allowing you to start/stop/restart in concurrent safe way. Schedule also instrument this Job and gather metrics and optionally expose them via uber-go/tally scope.

Install

go get github.com/sherifabdlnaby/sched

import "github.com/sherifabdlnaby/sched"

Requirements

Go 1.13 >=

Concepts

Job

Simply a func(){} implementation that is the schedule goal to run, and instrument.

Timer

An Object that Implements the type Timer interface{}. A Timer is responsible for providing a schedule with the next time the job should run and if there will be subsequent runs.

Packaged Implementations:

Fixed :- Infinitely Fires a job at a Fixed Interval (time.Duration)
Cron :- Infinitely Fires a job based on a Cron Expression, all Expressions supported by gorhill/cronexpr are supported.
Once :- A Timer that run ONCE after an optional specific delay or at a specified time, schedule will stop after it fires.

You can Implement your own Timer for your specific scheduling needs by implementing

type Timer interface {
	// done indicated that there will be no more runs.
    Next() (next time.Time, done bool)
}

Schedule

A Schedule wraps a Job and fires it according to Timer.

	fixedTimer30second, _ := sched.NewFixed(30 * time.Second)

	job := func() {
		log.Println("Doing some work...")
		time.Sleep(1 * time.Second)
		log.Println("Finished Work.")
	}

	// Create Schedule
	schedule := sched.NewSchedule("every30s", fixedTimer30second, job)

	// Start
	schedule.Start()

Options

Additional Options can be passed to Schedule to change its behavior.

// Create Schedule
schedule := sched.NewSchedule("every30s", fixedTimer30second, job,
	sched.WithLogger(sched.DefaultLogger()),
	opt2,
	opt3,
	....,
)

Logger Option

WithLogger( logger Logger) -> Supply the Schedule the Logger it is going to use for logging.

func DefaultLogger() Logger : Provide a Default Logging Interface to be used Instead of Implementing your own.
func NopLogger() Logger : A nop Logger that will not output anything to stdout.

Metrics Option

WithMetrics( scope tally.Scope) -> Supply the Schedule with a metrics scope it can use to export metrics.

Use any of uber-go/tally implementations (Prometheus, statsd, etc)

Use func WithConsoleMetrics(printEvery time.Duration) Option Implementation to Output Metrics to stdout (good for debugging)

Expected Runtime

WithExpectedRunTime(d time.Duration) -> Supply the Schedule with the expected duration for the job to run, schedule will output corresponding logs and metrics if job run exceeded expected.

Schedule(r)

Scheduler manage one or more Schedule creating them using common options, enforcing unique IDs, and supply methods to Start / Stop all schedule(s).

Exported Metrics

Metric	Type	Desc
up	Gauge	If the schedule is Running / Stopped
runs	Counter	Number of Runs Since Starting
runs_overlapping	Counter	Number of times more than one job was running together. (Overlapped)
run_actual_elapsed_time	Time	Elapsed Time between Starting and Ending of Job Execution
run_total_elapsed_time	Time	Total Elapsed Time between Creating the Job and Ending of Job Execution, This differ from Actual Elapsed time when Overlapping blocking is Implemented
run_errors	Counter	Count Number of Times a Job error'd(Panicked) during execution.
run_exceed_expected_time	Counter	Count Number of Times a Job Execution Time exceeded the Expected Time

In Prometheus Format

# HELP sched_run_actual_elapsed_time sched_run_actual_elapsed_time summary
# TYPE sched_run_actual_elapsed_time summary
sched_run_actual_elapsed_time{id="every5s",quantile="0.5"} 0.203843151
sched_run_actual_elapsed_time{id="every5s",quantile="0.75"} 1.104031623
sched_run_actual_elapsed_time{id="every5s",quantile="0.95"} 1.104031623
sched_run_actual_elapsed_time{id="every5s",quantile="0.99"} 1.104031623
sched_run_actual_elapsed_time{id="every5s",quantile="0.999"} 1.104031623
sched_run_actual_elapsed_time_sum{id="every5s"} 1.307874774
sched_run_actual_elapsed_time_count{id="every5s"} 2
# HELP sched_run_errors sched_run_errors counter
# TYPE sched_run_errors counter
sched_run_errors{id="every5s"} 0
# HELP sched_run_exceed_expected_time sched_run_exceed_expected_time counter
# TYPE sched_run_exceed_expected_time counter
sched_run_exceed_expected_time{id="every5s"} 0
# HELP sched_run_total_elapsed_time sched_run_total_elapsed_time summary
# TYPE sched_run_total_elapsed_time summary
sched_run_total_elapsed_time{id="every5s",quantile="0.5"} 0.203880714
sched_run_total_elapsed_time{id="every5s",quantile="0.75"} 1.104065614
sched_run_total_elapsed_time{id="every5s",quantile="0.95"} 1.104065614
sched_run_total_elapsed_time{id="every5s",quantile="0.99"} 1.104065614
sched_run_total_elapsed_time{id="every5s",quantile="0.999"} 1.104065614
sched_run_total_elapsed_time_sum{id="every5s"} 1.307946328
sched_run_total_elapsed_time_count{id="every5s"} 2
# HELP sched_runs sched_runs counter
# TYPE sched_runs counter
sched_runs{id="every5s"} 2
# HELP sched_runs_overlapping sched_runs_overlapping counter
# TYPE sched_runs_overlapping counter
sched_runs_overlapping{id="every5s"} 0
# HELP sched_up sched_up gauge
# TYPE sched_up gauge
sched_up{id="every5s"} 1

Examples

Inline Example

package main

import (
    "fmt"
    "github.com/sherifabdlnaby/sched"
    "log"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {

    cronTimer, err := sched.NewCron("* * * * *")
    if err != nil {
        panic(fmt.Sprintf("invalid cron expression: %s", err.Error()))
    }

    job := func() {
        log.Println("Doing some work...")
        time.Sleep(1 * time.Second)
        log.Println("Finished Work.")
    }

    // Create Schedule
    schedule := sched.NewSchedule("cron", cronTimer, job, sched.WithLogger(sched.DefaultLogger()))

    // Start Schedule
    schedule.Start()

    // Stop schedule after 5 Minutes
    time.AfterFunc(5*time.Minute, func() {
        schedule.Stop()
    })

    // Listen to CTRL + C
    signalChan := make(chan os.Signal, 1)
    signal.Notify(signalChan, syscall.SIGTERM, syscall.SIGINT, syscall.SIGQUIT)
    _ = <-signalChan

    // Stop before shutting down.
    schedule.Stop()

    return
}

Output for 3 minutes

2021-04-10T12:30: 13.132+0200    INFO    sched   sched/schedule.go: 96    Job Schedule Started    {"id": "cron"}
2021-04-10T12:30: 13.132+0200    INFO    sched   sched/schedule.go: 168   Job Next Run Scheduled  {"id": "cron", "After": "47s", "At": "2021-04-10T12:31:00+02:00"}
2021-04-10T12: 31: 00.000+0200    INFO    sched   sched/schedule.go: 168   Job Next Run Scheduled  {"id": "cron", "After": "1m0s", "At": "2021-04-10T12:32:00+02:00"}
2021-04-10T12: 31:00.000+0200    INFO    sched   sched/schedule.go: 193   Job Run Starting        {"id": "cron", "Instance": "8e1044ab-20b6-4acf-8a15-e06c0418522c"}
2021/04/10 12: 31: 00 Doing some work...
2021/04/10 12: 31: 01 Finished Work.
2021-04-10T12: 31: 01.001+0200    INFO    sched   sched/schedule.go: 208   Job Finished    {"id": "cron", "Instance": "8e1044ab-20b6-4acf-8a15-e06c0418522c", "Duration": "1.001s", "State": "FINISHED"}
2021-04-10T12:32: 00.002+0200    INFO    sched   sched/schedule.go: 168   Job Next Run Scheduled  {"id": "cron", "After": "1m0s", "At": "2021-04-10T12:33:00+02:00"}
2021-04-10T12: 32: 00.002+0200    INFO    sched   sched/schedule.go: 193   Job Run Starting        {"id": "cron", "Instance": "baae94eb-f818-4b34-a1f4-45b521a360a1"}
2021/04/10 12: 32: 00 Doing some work...
2021/04/10 12: 32: 01 Finished Work.
2021-04-10T12:32: 01.005+0200    INFO    sched   sched/schedule.go: 208   Job Finished    {"id": "cron", "Instance": "baae94eb-f818-4b34-a1f4-45b521a360a1", "Duration": "1.003s", "State": "FINISHED"}
2021-04-10T12: 33: 00.001+0200    INFO    sched   sched/schedule.go:168   Job Next Run Scheduled  {"id": "cron", "After": "1m0s", "At": "2021-04-10T12:34:00+02:00"}
2021-04-10T12:33: 00.001+0200    INFO    sched   sched/schedule.go: 193   Job Run Starting        {"id": "cron", "Instance": "71c8f0bf-3624-4a92-909c-b4149f3c62a3"}
2021/04/10 12: 33: 00 Doing some work...
2021/04/10 12: 33: 01 Finished Work.
2021-04-10T12: 33: 01.004+0200    INFO    sched   sched/schedule.go:208   Job Finished    {"id": "cron", "Instance": "71c8f0bf-3624-4a92-909c-b4149f3c62a3", "Duration": "1.003s", "State": "FINISHED"}

Output With CTRL+C

2021-04-10T12:28: 45.591+0200    INFO    sched   sched/schedule.go: 96    Job Schedule Started    {"id": "cron"}
2021-04-10T12:28: 45.592+0200    INFO    sched   sched/schedule.go: 168   Job Next Run Scheduled  {"id": "cron", "After": "14s", "At": "2021-04-10T12:29:00+02:00"}
2021-04-10T12: 29: 00.000+0200    INFO    sched   sched/schedule.go: 168   Job Next Run Scheduled  {"id": "cron", "After": "1m0s", "At": "2021-04-10T12:30:00+02:00"}
2021-04-10T12: 29:00.000+0200    INFO    sched   sched/schedule.go: 193   Job Run Starting        {"id": "cron", "Instance": "786540f1-594b-44a0-9a66-7181619e38a6"}
2021/04/10 12: 29: 00 Doing some work...
CTRL+C
2021-04-10T12: 29: 00.567+0200    INFO    sched   sched/schedule.go: 125   Stopping Schedule...    {"id": "cron"}
2021-04-10T12: 29: 00.567+0200    INFO    sched   sched/schedule.go: 130   Waiting active jobs to finish...        {"id": "cron"}
2021-04-10T12: 29: 00.567+0200    INFO    sched   sched/schedule.go: 171   Job Next Run Canceled   {"id": "cron", "At": "2021-04-10T12:30:00+02:00"}
2021/04/10 12: 29: 01 Finished Work.
2021-04-10T12: 29:01.000+0200    INFO    sched   sched/schedule.go: 208   Job Finished    {"id": "cron", "Instance": "786540f1-594b-44a0-9a66-7181619e38a6", "Duration": "1s", "State": "FINISHED"}
2021-04-10T12: 29: 01.000+0200    INFO    sched   sched/schedule.go: 133   Job Schedule Stopped {"id": "cron" }

Todo(s) and Enhancements

Control Logging Verbosity
Make Panic Recovery Optional
Make Job a func() error and allow retry(s), backoff, and collect errors and their metrics
Make Jobs context aware and support canceling Jobs Context.
Make allow Overlapping Optional and Configure How Overlapping is handled/denied.
Global Package-Level Metrics

License

Contribution

PR(s) are Open and Welcomed. ❤️

Crane scheduler is a Kubernetes scheduler which can schedule pod based on actual node load.

Crane-scheduler Overview Crane-scheduler is a collection of scheduler plugins based on scheduler framework, including: Dynamic scheuler: a load-aware

Dec 29, 2022

A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

Dec 17, 2022

Implement a logrus based logger

(Thanks for sending a pull request! Please make sure you click the link above to view the contribution guidelines, then fill out the blanks below.)

What does this implement/fix? Explain your changes.

this is a logrus based logger instead of using zap

Does this close any currently open issues?

…

Any relevant logs, error output, etc?

(If it’s long, please paste to https://ghostbin.com/ and insert the link here.)

Any other comments?

…

Where has this been tested?

…
Middleware Proof of Concept
Proof of Concept for Middleware

Just a simple Proof of Concept for Middleware...

A few things:

State handling needs to be stored just in the Schedule. I see some states in Schedule, and some in jobs (and for example, FINISHED state in Jobs, but the doco says a Finished State can not be rescheduled...)

the Schedule.transitionState is very simple now. It should handle:

[ ] Selectively Blocking on some State Transitions

[ ] Handle Panics (new State?)

[ ] Handle rescheduling Jobs (eg, so you could implement retry/backoff via middleware)

[ ] Handle Contexts when introduced?
NewJobWithID needed?

Just seems redundant as the Only Caller is NewJob?

https://github.com/sherifabdlnaby/sched/blob/6eeb2cc626e821489a11ee8219b239ced2f900a2/job/job.go#L32
Additions
Is your feature request related to a problem? Please describe. Wondering if you would be interested in the following PR's:

Methods to return individual Schedule's from the Scheduler by ID?

Return a slice of all Schedule ID's in the Scheduler

ExpectedRunTime to be calculated from previous runs in the Scheduler (maybe controllable via a Option?)

Callback function when we exceed the ExpectedRunTIme? (instead of just logging)

Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

🕰 Sched

Go In-Process Scheduler with Cron Expression Support

Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

Introduction

Install

Requirements

Concepts

Job

Timer

Schedule

Options

Logger Option

Metrics Option

Expected Runtime

Schedule(r)

Exported Metrics

In Prometheus Format

Examples

Inline Example

Output for 3 minutes

Output With CTRL+C

Todo(s) and Enhancements

License

Contribution

Owner

Sherif Abdel-Naby

Similar Resources

Comments

Implement a logrus based logger

What does this implement/fix? Explain your changes.

Does this close any currently open issues?

Any relevant logs, error output, etc?

Any other comments?

Where has this been tested?

Middleware Proof of Concept

NewJobWithID needed?

Additions

Related tags

Executes jobs in separate GO routines. Provides Timeout, StartTime controls. Provides Cancel all running job before new job is run.

Job worker service that provides an API to run arbitrary Linux processes.

gron, Cron Jobs in Go.

Lightweight, fast and dependency-free Cron expression parser (due checker) for Golang (tested on v1.13 and above)

Cloud-native, enterprise-level cron job platform for Kubernetes

This package provides the way to get the previous timestamp or the next timestamp that satisfies the cron expression.

Easily schedule commands to run multiple times at set intervals (like a cronjob, but with one command)

A lightweight job scheduler based on priority queue with timeout, retry, replica, context cancellation and easy semantics for job chaining. Build for golang web apps.

go-sche is a golang library that lets you schedule your task to be executed later.

Reminder is a Golang package to allow users to schedule alerts.