MySQL Backed Locking Primitive

go-mysql-lock

GoDoc Azure DevOps builds Azure DevOps coverage Go Report Card

go-mysql-lock provides locking primitive based on MySQL's GET_LOCK Lock names are strings and MySQL enforces a maximum length on lock names of 64 characters.

Use cases

Though there are mature locking primitives provided by systems like Zookeeper and etcd, when you have an application which is primarily dependent on MySQL for its uptime and health, added resiliency provided by systems just mentioned doesn't add much benefit. go-mysql-lock helps when you have multiple application instances which are backed by a common mysql instance and you want only one of those application instances to hold a lock and do certain tasks.

Installation

go get github.com/sanketplus/go-mysql-lock

Example:

package main

import (
    "context"
    "database/sql"
    
    _ "github.com/go-sql-driver/mysql"
    "github.com/sanketplus/go-mysql-lock"
)

func main() {
	db, _ := sql.Open("mysql", "root@tcp(localhost:3306)/dyno_test")

	locker := gomysqllock.NewMysqlLocker(db)

	lock, _ := locker.Obtain("foo")
	lock.Release()
}

Features

Customizable Refresh Period

Once the lock is obtained, a goroutine periodically (default every 1 second) keeps pinging on connection since the lock is valid on a connection(session). To configure the refresh interval

locker := gomysqllock.NewMysqlLocker(db, gomysqllock.WithRefreshInterval(time.Millisecond*500))

Obtain Lock With Context

By default, an attempt to obtain a lock is backed by background context. That means the Obtain call would block indefinitely. Optionally, an Obtain call can be made with user given context which will get cancelled with the given context.

The following call will give up after a second if the lock was not obtained.

locker := gomysqllock.NewMysqlLocker(db)
ctxShort, _ := context.WithDeadline(context.Background(), time.Now().Add(time.Second))
lock, err := locker.ObtainContext(ctxShort, "key")

Obtain Lock With (MySQL) Timeout

MySQL has the ability to timeout and return if the lock can't be acquired in a given number of seconds. This timeout can be specified when using ObtainTimeout and ObtainTimeoutContext. On timeout, ErrMySQLTimeout is returned, and the lock is not obtained.

The following call will give up after a second if the lock was not obtained, using MySQL timeout option:

locker := gomysqllock.NewMysqlLocker(db)
lock, err := locker.ObtainTimeout("key", 1)

Know When The Lock is Lost

Obtained lock has a context which is cancelled if the lock is lost. This is determined while a goroutine keeps pinging the connection. If there is an error while pinging, assuming connection has an error, the context is cancelled. And the lock owner gets notified of the lost lock.

context := lock.GetContext()

Compatibility

This library is tested (automatically) against MySQL 8 and MariaDB 10.1, and it should work for MariaDB versions >= 10.1 and MySQL versions >= 5.6.

Note that GET_LOCK function won't lock indefinitely on MariaDB 10.1 / MySQL 5.6 and older, as 0 or negative value for timeouts are not accepted in those versions. This means that in MySQL <= 5.6 / MariaDB <= 10.1 you can't use Obtain or ObtainContext. To achieve a similar goal, you can use ObtainTimeout (and ObtainTimeoutContext) using a very high timeout value.

Owner
Sanket Patel
DevOps for Fun and Profit!™
Sanket Patel
Comments
  • Fix Vitess support by removing use of `COALESCE()`

    Fix Vitess support by removing use of `COALESCE()`

    Vitess supports GET_LOCK() and RELEASE_LOCK() but does not allow you to call these functions within other functions like COALESCE(). This PR switches back to calling GET_LOCK() by itself but tests for a NULL result instead of coalescing it. This change retains the same compatibility with other MySQL/MariaDB versions while adding Vitess support, so it should be a good compromise to support the widest set of options.

    You can see an example here from trying these queries against a Vitess 14 test instance, and can see how the vtgate will fail the COALESCE(GET_LOCK(...

    mysql> SELECT COALESCE(GET_LOCK('foo', 1), 2) FROM dual;
    ERROR 1235 (42000): get_lock(:vtg1, :vtg2) allowed only with dual
    mysql> SELECT GET_LOCK('foo', 1);
    +--------------------+
    | get_lock('foo', 1) |
    +--------------------+
    |                  1 |
    +--------------------+
    1 row in set (0.01 sec)
    
    mysql> DO RELEASE_LOCK('foo');
    Query OK, 0 rows affected (0.01 sec)
    
  • Add MySQL timeout parameter / fix compatibility with MariaDB 10.1 and older

    Add MySQL timeout parameter / fix compatibility with MariaDB 10.1 and older

    I added two methods to use an explicit timeout, as they might be useful, and this also fixes an incompatibility with older MySQL/MariaDB (tested with 10.1) where GET_LOCK does not support -1 for infinite value.

    I added new test cases for the timeout part. I tested it with a build flag (as in tests with older versions current tests will fail on -1 timeouts), but I'm open to suggestions :-)

  • Go Routine Leak

    Go Routine Leak

    There is currently a go-routine leak when spawning a refresher goroutine go lock.refresher(l.refreshInterval, cancelFunc) due to the use of an unbuffered channel unlocker: make(chan struct{}) , as in l.Release() called after connection ping fails due to cancellation of connection context, as there is no consumer listening to the channel l.unlocker <- struct{}{} while calling release goes to an infinite deadlock wait.

    Using unlocker: make(chan struct{}, 1) should fix this deadlock.

  • Would it be possible create a v0.0.5 tag based of the current state of master?

    Would it be possible create a v0.0.5 tag based of the current state of master?

    Hello,

    Would it be possible to create a v0.0.5 tag from the current state of master. The company I work at prefers to use tags in our module versions rather than using whatever is on master and we want to use some of the improved error handling that is in master but not in v0.0.4

    Thanks

  • DB connection not returned to connection pool if failed to acquire the lock

    DB connection not returned to connection pool if failed to acquire the lock

    I got my application hung after doing a load test. I dug into the library code and see that the database connection is not closed (return connection to the connection pool) if unable to obtain the lock.

  • Multiple Retries Before Giving Up

    Multiple Retries Before Giving Up

    Currently, a goroutine periodically pings on the connection that holds the lock. If a ping fails, it bails out and closes the connection.

    1. Confirm what happens to connection when a ping fails
    2. If connection is still kept in pool, consider multiple ping retries before giving up (and possibly make it configurable)
Lockgate is a cross-platform locking library for Go with distributed locks using Kubernetes or lockgate HTTP lock server as well as the OS file locks support.

Lockgate Lockgate is a locking library for Go. Classical interface: 2 types of locks: shared and exclusive; 2 modes of locking: blocking and non-block

Dec 16, 2022
A distributed locking library built on top of Cloud Spanner and TrueTime.

A distributed locking library built on top of Cloud Spanner and TrueTime.

Sep 13, 2022
An implementation of a distributed KV store backed by Raft tolerant of node failures and network partitions 🚣
An implementation of a distributed KV store backed by Raft tolerant of node failures and network partitions 🚣

barge A simple implementation of a consistent, distributed Key:Value store which uses the Raft Concensus Algorithm. This project launches a cluster of

Nov 24, 2021
A distributed MySQL binlog storage system built on Raft
A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Dec 31, 2022
Go-mysql-orm - Golang mysql orm,dedicated to easy use of mysql

golang mysql orm 个人学习项目, 一个易于使用的mysql-orm mapping struct to mysql table golang结构

Jan 7, 2023
A playground project to create a simple web API backed by a MySQL datastore.

A playground project to create a simple web API backed by a MySQL datastore. Which will allow evaluating ORM & HTTP router Go modules.

Oct 16, 2021
Simplified distributed locking implementation using Redis

redislock Simplified distributed locking implementation using Redis. For more information, please see examples. Examples import ( "fmt" "time"

Dec 24, 2022
Lockgate is a cross-platform locking library for Go with distributed locks using Kubernetes or lockgate HTTP lock server as well as the OS file locks support.

Lockgate Lockgate is a locking library for Go. Classical interface: 2 types of locks: shared and exclusive; 2 modes of locking: blocking and non-block

Dec 16, 2022
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).

goInterLock is golang job/task scheduler with distributed locking mechanism. In distributed system locking is preventing task been executed in every instant that has the scheduler,

Dec 5, 2022
Multithreaded key value pair store using thread safe locking mechanism allowing concurrent reads
Multithreaded key value pair store using thread safe locking mechanism allowing concurrent reads

Project Amnesia A Multi-threaded key-value pair store using thread safe locking mechanism allowing concurrent reads. Curious to Try it out?? Check out

Oct 29, 2022
A distributed locking library built on top of Cloud Spanner and TrueTime.

A distributed locking library built on top of Cloud Spanner and TrueTime.

Sep 13, 2022
Filesystem based locking for golang

go-fs-lock Filesystem based locking Lead Maintainer Steven Allen Table of Contents Install Usage Contribute License Install go-fs-lock is a standard G

Jan 18, 2022
Null Types, Safe primitive type conversion and fetching value from complex structures.

Typ Typ is a library providing a powerful interface to impressive user experience with conversion and fetching data from built-in types in Golang Feat

Sep 26, 2022
Fast resizable golang semaphore primitive

semaphore Fast resizable golang semaphore based on CAS allows weighted acquire/release; supports cancellation via context; allows change semaphore lim

Dec 13, 2022
Scan database/sql rows directly to structs, slices, and primitive types

Scan Scan standard lib database rows directly to structs or slices. For the most comprehensive and up-to-date docs see the godoc Examples Multiple Row

Dec 28, 2022
A faster RWLock primitive in Go, 2-3 times faster than RWMutex. A Go implementation of concurrency control algorithm in paper

Go Left Right Concurrency A Go implementation of the left-right concurrency control algorithm in paper <Left-Right - A Concurrency Control Technique w

Jan 6, 2023
Simple & Primitive multi client communication system

What is this Simple & Primitive multi client communication system. e.g. chat system for larning Supported Broadcast message Unicast message Not Suppor

Dec 3, 2021
Slice conversion between primitive types

sliceconv Sliceconv implements conversions to and from string representations of primitive types on entire slices. The package supports types int, flo

Sep 27, 2022
Package iter provides generic, lazy iterators, functions for producing them from primitive types, as well as functions and methods for transforming and consuming them.

iter Package iter provides generic, lazy iterators, functions for producing them from primitive types, as well as functions and methods for transformi

Dec 16, 2022
Go module that provides primitive functional programming utilities.

Functional Functional provides a small set of pure functions that are common in functional programming languages, such as Reduce, Map, Filter, etc. Wi

Jun 12, 2022