An extremely fast UUID alternative written in golang

Overview

  • WUID is a globally unique number generator, while it is NOT a UUID implementation.
  • WUID is 10-135 times faster than UUID and 4600 times faster than generating unique numbers with Redis.
  • Each WUID instance generates unique 64-bit integers in sequence. The high 28 bits are loaded from a data store. By now, Redis, MySQL, MongoDB and Callback are supported.

Benchmarks

BenchmarkWUID       187500764            6.38 ns/op        0 B/op          0 allocs/op
BenchmarkRand       97180698            12.2 ns/op         0 B/op          0 allocs/op
BenchmarkTimestamp  17126514            67.8 ns/op         0 B/op          0 allocs/op
BenchmarkUUID_V1    11986558            99.6 ns/op         0 B/op          0 allocs/op
BenchmarkUUID_V2    12017754           101 ns/op           0 B/op          0 allocs/op
BenchmarkUUID_V3     4925020           242 ns/op         144 B/op          4 allocs/op
BenchmarkUUID_V4    14184271            84.1 ns/op        16 B/op          1 allocs/op
BenchmarkUUID_V5     4277338           274 ns/op         176 B/op          4 allocs/op
BenchmarkRedis         35462         35646 ns/op         176 B/op          5 allocs/op
BenchmarkSnowflake   4931476           244 ns/op           0 B/op          0 allocs/op
BenchmarkULID        8410358           141 ns/op          16 B/op          1 allocs/op
BenchmarkXID        15000969            79.2 ns/op         0 B/op          0 allocs/op

Features

  • Extremely fast
  • Thread-safe
  • Being unique across time
  • Being unique within a data center
  • Being unique globally if all data centers share a same data store, or they use different section IDs
  • Being capable of generating 100M unique numbers in a single second with each WUID instance
  • Auto-renew when the low 36 bits are about to run out

Install

go get -u github.com/edwingeng/wuid

Usage examples

Redis

import "github.com/edwingeng/wuid/redis/wuid"

newClient := func() (redis.Cmdable, bool, error) {
    var client redis.Cmdable
    // ...
    return client, true, nil
}

// Setup
g := NewWUID("default", nil)
_ = g.LoadH28FromRedis(newClient, "wuid")

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", g.Next())
}

MySQL

import "github.com/edwingeng/wuid/mysql/wuid"

newDB := func() (*sql.DB, bool, error) {
    var db *sql.DB
    // ...
    return db, true, nil
}

// Setup
g := NewWUID("default", nil)
_ = g.LoadH28FromMysql(newDB, "wuid")

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", g.Next())
}

MongoDB

import "github.com/edwingeng/wuid/mongo/wuid"

newClient := func() (*mongo.Client, bool, error) {
    var client *mongo.Client
    // ...
    return client, true, nil
}

// Setup
g := NewWUID("default", nil)
_ = g.LoadH28FromMongo(newClient, "test", "wuid", "default")

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", g.Next())
}

Callback

import "github.com/edwingeng/wuid/callback/wuid"

// Setup
g := NewWUID("default", nil)
_ = g.LoadH28WithCallback(func() (int64, func(), error) {
    resp, err := http.Get("https://stackoverflow.com/")
    if resp != nil {
        defer func() {
            _ = resp.Body.Close()
        }()
    }
    if err != nil {
        return 0, nil, err
    }

    bytes, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return 0, nil, err
    }

    fmt.Printf("Page size: %d (%#06x)\n\n", len(bytes), len(bytes))
    return int64(len(bytes)), nil, nil
})

// Generate
for i := 0; i < 10; i++ {
    fmt.Printf("%#016x\n", g.Next())
}

Mysql table creation

CREATE TABLE IF NOT EXISTS `wuid` (
    `h` int(10) NOT NULL AUTO_INCREMENT,
    `x` tinyint(4) NOT NULL DEFAULT '0',
    PRIMARY KEY (`x`),
    UNIQUE KEY `h` (`h`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Section ID

You can specify a custom section ID for the generated numbers with wuid.WithSection when you call wuid.NewWUID. The section ID must be in between [1, 7].

Best practices

  • Use different keys/tables/docs for different purposes.
  • Pass a logger to wuid.NewWUID and keep an eye on the warnings that include "renew failed", which means that the low 36 bits are about to run out in hours to hundreds of hours, and WUID fails to get a new number from your data store.

Special thanks

Similar Resources

Generate, encode, and decode UUIDs v1 with fast or cryptographic-quality random node identifier.

A Go package for generating and manipulating UUIDs Generate, encode, and decode UUIDs v1, as defined in RFC 4122, in Go. Project Status v1.1.0 Stable:

Sep 27, 2022

A UUIDv4 generation package written in go

Goid A Go package to generate V4 UUIDs Documentation The API docs can be viewed here or generated with godoc. Usage An example of generating a v4 UUID

Dec 24, 2022

A simple to use Go (golang) package to generate or parse Twitter snowflake IDs

snowflake snowflake is a Go package that provides A very simple Twitter snowflake generator. Methods to parse existing snowflake IDs. Methods to conve

Dec 30, 2022

❄ An Lock Free ID Generator for Golang based on Snowflake Algorithm (Twitter announced).

❄ An Lock Free ID Generator for Golang based on Snowflake Algorithm (Twitter announced).

❄ An Lock Free ID Generator for Golang based on Snowflake Algorithm (Twitter announced).

Dec 14, 2022

✨ Generate unique IDs (Port of Node package "generate-snowflake" to Golang)

✨ Generate Snowflake Generate unique IDs. Inspired by Twitter's Snowflake system. 📦 Installation Initialize your project (go mod init example.com/exa

Feb 11, 2022

Golang wrapper for the Snowflake Api.

GoSnowflakeApi GoSnowflakeApi is an api wrapper for the snowflake api written in golang for golang developers. Example package main import ( "fmt

Jul 25, 2021

Snowflake implemented in GO (Golang)

snowflake Snowflake is a fast, goroutine-safe unique ID generator built for distributed systems Key concepts Snowflake Snowflakes are int64s. uint64 i

Oct 27, 2022

Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm

Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm

go-orca Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm Disclaimer This project is under active development and i

Nov 22, 2022

Snowflake - A simple to use Go (golang) package to generate or parse Twitter snowflake IDs

Snowflake - A simple to use Go (golang) package to generate or parse Twitter snowflake IDs

❄️ Go-Snowflake A Snowflake Generator for Go A simple to use Go (golang) package

Oct 20, 2022
Comments
  • 请问一下,如果不同机房,通过设置Section ID可以实现跨机房全局唯一吗

    请问一下,如果不同机房,通过设置Section ID可以实现跨机房全局唯一吗

    // A机房
    once.Do(func() {
    	g=wuid.NewWUID("default",nil,wuid.WithSection(1))
            g.LoadH24FromMysql(....
    })
    // B机房
    once.Do(func() {
    	g=wuid.NewWUID("default",nil,wuid.WithSection(2))
            g.LoadH24FromMysql(....
    })
    // 然后g在程序中作为全局变量,这样,两个g都不共享mysql,可以达到全局唯一吗
    
  • add Postgres support

    add Postgres support

    • [x] add Postgres support using mysql as template
    • [x] Tested using Postgres on Docker
    • [x] Automated testing possible using script

    ../wuid/pgsql/dbtest.sh

    • [x] Add readme to Mysql and Postgres packages

    dbtest.sh Test Result:

    ~/Dev/golang/src/github.com/dwin/wuid/pgsql ./testdb.sh
    baaf8b92b2ae54ae3e47e0731f93655aed67a7b61f63fe673e0dc27a07940980
    wuid-postgres
    wuid-postgres
     - TestLoadH24FromPg complete -
     - TestLoadH24FromPgWithOpts complete -
     - TestWUID_LoadH24FromPg_UserPass complete -
     - TestWUID_Next_Renew complete -
     - TestWithSection complete -
    2018/06/08 13:33:21 <wuid> new h24: 1505
     - BenchmarkLoadH24FromPg complete -
    goos: darwin
    goarch: amd64
    pkg: github.com/dwin/wuid/pgsql
    BenchmarkLoadH24FromPg-4   	2018/06/08 13:33:22 <wuid> new h24: 1506
     - BenchmarkLoadH24FromPg complete -
    2018/06/08 13:33:22 <wuid> new h24: 1507
     - BenchmarkLoadH24FromPg complete -
    2018/06/08 13:33:22 <wuid> new h24: 1508
     - BenchmarkLoadH24FromPg complete -
    2018/06/08 13:33:22 <wuid> new h24: 1509
     - BenchmarkLoadH24FromPg complete -
    2018/06/08 13:33:22 <wuid> new h24: 1510
     - BenchmarkLoadH24FromPg complete -
    2018/06/08 13:33:23 <wuid> new h24: 1511
     - BenchmarkLoadH24FromPg complete -
    200000000	         8.92 ns/op
    PASS
    coverage: 87.3% of statements
    ok  	github.com/dwin/wuid/pgsql	37.337s
    wuid-postgres
    wuid-postgres
    
  • 调用wuid生成id后,会自动关闭数据库连接,导致后续数据库操作无法继续执行

    调用wuid生成id后,会自动关闭数据库连接,导致后续数据库操作无法继续执行

    调用wuid生成id后,会自动关闭数据库连接,导致后续数据库操作无法继续执行 package tools import (

    "database/sql"
    "github.com/astaxie/beego/orm"
    "github.com/edwingeng/wuid/mysql"
    _"github.com/go-sql-driver/mysql"
    

    )

    var newDB = func() (*sql.DB,bool,error){ db,_ := orm.GetDB() return db,true,nil }

    var g = wuid.NewWUID("default", nil)

    func NewId() (uint64){ g.LoadH28FromMysql(newDB, "wuid") return g.Next() }

    调用代码:

    func AddUser(u *User) (int64, error) { // db,err := orm.GetDB() //fmt.Println(err) o := orm.NewOrm() u.Id = 74837477878 for i:=0;i<100;i++ { fmt.Println(tools.NewId()) }

    fmt.Println(u)
    return o.Insert(u)
    

    }

    插入操作无法继续执行, [ORM]2019/05/12 22:09:56 -[Queries/default] - [FAIL / db.Exec / 0.0ms] - [INSERT INTO sys_user (id, username, password, phone, email, name, avatar, salt, status, is_staff, last_login_time, login_error_times, remark, delete_flag, create_by, create_time, update_by, update_time) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)] - 74837477878, qingmumao, xxxxxxxx, 13332978703, [email protected], ,, , `0`, `0`, `0`, `0`,, 0, 0, 0, 0, 0 - sql: database is closed

  • Bump go.mongodb.org/mongo-driver from 1.3.3 to 1.5.1

    Bump go.mongodb.org/mongo-driver from 1.3.3 to 1.5.1

    Bumps go.mongodb.org/mongo-driver from 1.3.3 to 1.5.1.

    Release notes

    Sourced from go.mongodb.org/mongo-driver's releases.

    MongoDB Go Driver 1.5.1

    The MongoDB Go driver team is pleased to release 1.5.1 of the official Go driver.

    This release contains several bug fixes. Due to the issue below, we recommend all users upgrade to this version of the driver.

    Documentation can be found on pkg.go.dev and the MongoDB documentation site. BSON library documentation is also available on pkg.go.dev. Questions and inquiries can be asked on the MongoDB Developer Community. Bugs can be reported in the Go Driver Jira where a list of current issues can be found.

    This CVE describes a security issue with the driver's BSON marshalling system. BSON marshalling functions would incorrectly handle null bytes embedded in BSON key names and the pattern/options fields of a BSON regex value. BSON marshalling functions now correctly validate and error if there is an embedded null byte in BSON key names or the pattern/options fields of a BSON regex value. We recommend all users of the driver upgrade to this version.

    CVE ID: CVE-2021-20329 Title: Specific cstrings input may not be properly validated in the MongoDB Go Driver Description: Specific cstrings input may not be properly validated in the MongoDB Go Driver when marshalling Go objects into BSON. A malicious user could use a Go object with specific string to potentially inject additional fields into marshalled documents. This issue affects all MongoDB GO Drivers up to (and including) 1.5.0. CVSS score: 6.8 CVSS:3.1/AV:N/AC:H/PR:L/UI:N/S:U/C:H/I:H/A:N Affected products and versions, MongoDB Go Driver versions <= 1.5.0 Underlying operating systems affected: All

    For a full list of tickets included in this release, please see the links below:

    Bugs

    Tasks

    MongoDB Go Driver 1.5.0

    The MongoDB Go driver team is pleased to release 1.5.0 of the official Go driver.

    This release contains several new features and usability improvements for the driver.

    Documentation can be found on pkg.go.dev and the MongoDB documentation site. BSON library documentation is also available on pkg.go.dev. Questions and inquiries can be asked on the MongoDB Developer Community. Bugs can be reported in the Go Driver Jira where a list of current issues can be found.

    This release contains a new errors API for the primary mongo package. Users can now detect duplicate key errors, timeouts, and network errors via the mongo.IsDuplicateKeyError, mongo.IsTimeout, and mongo.IsNetworkError functions, respectively. Additionally, a new UpdateByID function has been added to the mongo.Collection type to update a single document with a given _id value.

    The Go Driver now supports using GCP and Azure key management services with the client-side field level encryption feature. In addition, AWS key management support has been enhanced to allow authenticating with temporary AWS credentials. See the MongoDB docs for more information about these improvements. Use of client-side field level encryption requires users to install the latest released version of libmongocrypt. Note: This means that existing applications that use this feature will need to upgrade the libmongocrypt dependency when upgrading to this driver version; otherwise, the application will fail to compile. Users can upgrade to the latest development release of libmongocrypt via the OS-specific instructions for macos, Windows, and Linux.

    Monitoring has now been added for various server events. A ServerMonitor set on a mongo.Client monitors changes on the MongoDB deployment it is connected to and reports the changes in the client's representation of the deployment.

    The driver will now error if a map with more than one key is used as a hint option, sort option, or for index creation. This is to prevent unexpected behavior, for example, an index being created with the keys in the wrong order.

    ... (truncated)

    Commits
    • 40c0e70 Update version to v1.5.1
    • 3a89e6c GODRIVER-1923 Error if BSON cstrings contain null bytes (#622)
    • 1a2534c GODRIVER-1935 Update scram/stringprep dependencies (#624)
    • 6ea353a GODRIVER-1918 Check for zero length in readstring (#613)
    • d5e11aa GODRIVER-1919 Support decoding ObjectIDs from hex strings in BSON (#610)
    • e0ed6d6 Update version to v1.5.1+prerelease
    • 6760875 Update version to v1.5.0
    • 19a368c GODRIVER-1911 Fix Windows/macos test failures for CSFLE (#603)
    • 2a5f9a4 GODRIVER-1879 Apply connectTimeoutMS to TLS handshake (#594)
    • 2c5b75b GODRIVER-1855 Support AWS authentication with temporary credentials in CSFLE ...
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

Related tags
A UUID package originally forked from github.com/satori/go.uuid

UUID Package uuid provides a pure Go implementation of Universally Unique Identifiers (UUID) variant as defined in RFC-4122. This package supports bot

Dec 30, 2022
Alternative random id generation to uuid

randid import "go.withmatt.com/randid" id := randid.New().String() randid's goals are simple: a smaller, more compact, faster version of uuid4. I don

Nov 16, 2022
UUID package for Golang

UUID package for Go language This package provides pure Go implementation of Universally Unique Identifier (UUID). Supported both creation and parsing

Dec 9, 2021
UUID package for Go

UUID package for Go language This package provides pure Go implementation of Universally Unique Identifier (UUID). Supported both creation and parsing

Jan 2, 2023
A lightweight UUID implementation

uuid uuid is a lightweight implementation for Univerally unique identifier. Usage var id UUID = uuid.Rand() fmt.Println(id.Hex()) fmt.Println(id.Raw()

Feb 25, 2020
Library to integrate github.com/google/uuid with gopkg.in/vmihailenco/msgpack

Library to integrate github.com/google/uuid with gopkg.in/vmihailenco/msgpack

Apr 26, 2022
A simple uuid library based on RFC 4122

UUID generator A simple library that generates uuids. Supported versions: version 1 version 3 version 4 version 5 Supported variants: DCE Microsoft Th

Oct 20, 2021
Generate UUID for script's sql

Generate UUID for script's sql Instale o go em sua maquina https://golang.org/doc/install Baixe as seguintes dependecias go get github.com/satori/go.u

Oct 26, 2021
A tiny and fast Go unique string generator

Nano ID A tiny and fast Go unique string generator Safe. It uses cryptographically strong random APIs and tests distribution of symbols. Compact. It u

Nov 11, 2022
Compact, sortable and fast unique IDs with embedded metadata.
Compact, sortable and fast unique IDs with embedded metadata.

A spec for unique IDs in distributed systems based on the Snowflake design, i.e. a coordination-based ID variant. It aims to be friendly to both machi

Dec 22, 2022