Fast, scalable pseudo random number generator based on xxh3

kelindar/xxrand
Go Version PkgGoDev Go Report Card License Coverage

XXH3-Based Pseudorandom Number Generator

This package contains an experimental implementation of a noise based pseudorandom number generator that scales with multiple CPUs and from my benchmarks around 10x faster than using math.Rand. It uses xx3 algorithm to hash a counter, the default counter being the time stamp counter (using RDTSC instruction). That being said, most of use-cases probably are better of with using math.Rand since it should provide better randomness characteristics.

Features

  • Optimized to scale on multiple CPUs when called without the state (e.g. Int31n()).
  • Supports most of math/rand functions.
  • Roughly 10x faster than math/rand equivalent.

What is this for?

  • You can use this in the benchmarks where where math/rand tends to create a lock contention, especially when you are benchmarking multiple goroutines in parallel and generating some random data as you go.
  • You can use this in building games with Go, where performance is critical and potentially you are generating a lot of random numbers.
  • You can use this for use-cases that do not require extremely random numbers, such as load-balancing for example.

Benchmark Results

cpu: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz
BenchmarkParallel/rand-001-8            394602625                3.023 ns/op           0 B/op          0 allocs/op
BenchmarkParallel/rand-008-8            360640208                3.130 ns/op           0 B/op          0 allocs/op
BenchmarkParallel/rand-032-8            368496836                3.492 ns/op           0 B/op          0 allocs/op
BenchmarkParallel/rand-128-8            392360864                3.041 ns/op           0 B/op          0 allocs/op
BenchmarkParallel/rand-512-8            388266206                3.043 ns/op           0 B/op          0 allocs/op
BenchmarkParallel/rand-2048-8           382660137                3.097 ns/op           0 B/op          0 allocs/op
Owner
Roman Atachiants
As a hacker with a Ph.D., I'm scaling backend services for half a billion people in the Middle East.
Roman Atachiants
Similar Resources

Encriptator using random generated numbers

public-private-key-encrypter Encriptator using random generated numbers The input file must be in one file called 'data.txt' The execution will genera

Oct 15, 2021

Just some random matchers

Just some random matchers

The package provides a matcher interface to match a given value of any types.

Nov 3, 2022

generate random data like name, email, uuid, address, images and etc.

gg-rand generate random data like name, email, uuid, address, images and etc. build and install: make run: gg-rand $ gg-rand SillyName : Knavesa

Nov 16, 2022

Generate some random data

fake-data-generator-api generate some random data installing and using

Dec 2, 2022

Generates a random alphanumeric string of a given length.

randstring randstring.Create () is fast and has minimal memory allocation. It returns a random alphanumeric string of a given length. Install go get g

Jan 7, 2022

GoApiRandom - Api to get random numbers

GoApiRandom - Api to get random numbers

Jan 18, 2022

Print random bytes from a secure source to stdout.

Print random bytes from a secure source to stdout.

Feb 11, 2022

A simple API for computing diffs of your documents over the time built on a scalable technology stack.

Diffme API WIP - this is an API to compute diffs between documents. It serves as a way to easily create audit logs for documents in your system, think

Sep 8, 2021

Scalable golang ratelimiter using the sliding window algorithm. Currently supports only Redis.

Scalable golang ratelimiter using the sliding window algorithm. Currently supports only Redis.

go-ratelimiter Scalable golang ratelimiter using the sliding window algorithm. Currently supports only Redis. Example usage client := redis.NewClient

Oct 19, 2021
Comments
  • Other fast(er) hash algorithms

    Other fast(er) hash algorithms

    Just out of curiosity, let me mention there are faster alternatives to xxh3 or algos with about the same speed but jaw-droppingly simpler. See https://github.com/rurban/smhasher/ .

    Namely fastest (for both bulk & short inputs) seems to be wyhash and simpliest from the fastest ones mx3 (btw. I feel mx3 could still have some space for speed optimization as I suspect some false dependencies between instructions in the pipeline... - maybe even good old manual loop unrolling could make a difference, IDK).

    But maybe xxh3 is better in some ways which I don't see but would like to. I can imagine something related to DBs but I don't know what exactly :wink:.

Related tags
A tool to determine the highest version number that's smaller than a target version number

semver-highest A tool to determine the highest version number that's smaller than a target version number. Installation go install github.com/marten-s

Oct 13, 2021
Fast and scalable pseudorandom generator for Go

fastrand Fast pseudorandom number generator. Features Optimized for speed. Performance scales on multiple CPUs. How does it work? It abuses sync.Pool

Dec 22, 2022
Daypaper sets your GNOME wallpaper based on the time of day from a random and relevant Unsplash image.

Daypaper Daypaper sets your GNOME wallpaper based on the time of day from a random and relevant Unsplash image. Installation You will need an Access T

May 23, 2022
Generates random text based on trigrams generated from input text
Generates random text based on trigrams generated from input text

Trigrams Generates random text based on trigrams generated from input text Contents Building Running Using Implementation notes NGram size Maximum wor

Feb 9, 2022
Go linter that warns about number of arguments in functions.

argslen linter Argslen is a linter that checks for long list of argument in functions. The default limit is 5 (maxArguments) and skip the test files (

Sep 17, 2022
Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.
Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Jan 2, 2023
get windows system call number dynamically, it doesn't need the system call table.

GetSyscallID get windows system call number dynamically, it doesn't need the system call table. Try download go get github.com/akkuman/getsyscallid/cm

Jun 1, 2022
Exercise #8: Phone Number Normalizer
Exercise #8: Phone Number Normalizer

Exercise #8: Phone Number Normalizer Exercise details This exercise is fairly st

Jan 9, 2022
randstr is a module that contains functions for generating random strings.

randstr is a module that contains functions for generating random strings. The functions in this module uses the crypto/rand package. Installa

Nov 13, 2021
✔️ Get random data for your app from a third-party source.

Random Data Securely produced random data for application testing. FAQ What would i use this data for? You can use this information to test your apps

Jul 5, 2022