This change introduces three common Golang patterns to the SonyFlake project test framework:

• types: this will allow fine-tuned control over imported types by defining how they will be used, project-wide

• mock: this allows the generation of mock constructors, which lends itself to testing any individual path in a function by "injecting" a mock object which matches the expected type

• array-based-testing: privateIPv4 and lower16BitPrivateIP were lacking tests for their error paths, but using types and mocks, I was able add two array-based tests that cover all test cases for these methods

This change also increases test coverage to 💯, while adding no additional time for testing ⌛ .

This is a really cool project, and I'd be happy to contribute more!

Just curious to know, how practical is it to limit generation of 2^8 (256) unique IDs per 10ms on databases. Can someone please explain me with example like if I have 4GB RAM to 16GB RAM database instance, how many insert load I can expect to be handled by that server?

It will help me use sonyflake as id generation at database level.

Hi!

We would like to know if sonyflakes generated from now are a guaranteed sort order and if not then when is the next time that length will change. We have noticed an issue in our database where our sort order is being ill-computed based on that we sometimes store sonyflakes as strings. Could you kindly affirm whether the length of a sonyflake string can change? Or is there a guaranteed length.

In particular the following algorithm...

l = [
    "1",
    "9999",
    "2",
    "10000",
    "3",
    "4",
    "999",
    "6",
    "7",
    "8",
    "9",
    "5",
    "10",
    "100",
    "1000",
    "90",
]

print(l)
l.sort()
print(l)

Shows that say the number 2 > 1000000000000000 when doing string comparison. I would like to know how Sonyflakes generated reliably are affected by this as the impact of this could affect the sort ordering for some production data (may require migration). Due to the algorithmic complexity of the Sonyflake and how its generated (sonyflakes are always the same length I believe) I'm not sure if we would be impacted by this in the near future.

Go example

package main

import (
    "fmt"
    "sort"
)

func main() {
    sl := []string{"1", "9999", "2", "10000", "3", "4", "999", "6", "7", "8", "9", "5", "10", "100", "1000", "90"}
    fmt.Println(sl)
    sort.Strings(sl)
    fmt.Println(sl)
}

I appreciate that this is not your concern and represents a mistake made in our systems, however I would appreciate any help on the matter. It would be helpful if I can run unit tests locally to generate snowflake now to the range generate one years in the future, but I have no idea how to adjust the snowflake without changing my system time like this. Appreciate the easiest way to diagnose the impact!

Hi,

Thanks for the library. It would be nice to get the time{} struct the sonyflake is based off so that implementing services can prevent drift between recording current time and time allocated to the sonyFlake.

Please let me know if this is something you can do.

Cheers!

$ go test  -bench=.
sonyflake id: 838861161
decompose: map[id:838861161 machine-id:361 msb:0 sequence:0 time:50]
max sequence: 255
number of id: 256000
number of cpu: 8
number of id: 100000
goos: linux
goarch: amd64
pkg: github.com/sony/sonyflake
BenchmarkNextID-8            	   50000	     38984 ns/op
--- FAIL: BenchmarkNextID_validate
    sonyflake_test.go:205: NextID returned err:over the time limit
FAIL
exit status 1
FAIL	github.com/sony/sonyflake	16.740s

Fix ineffassign error. And, add a badge image.

% pwd
/path/to/sonyflake

% ineffassign .
/path/to/sonyflake/awsutil/awsutil.go:59:5: err assigned and not used

Seems like when I try to use the sonyflake.Decompose on the generated ID, I get a timestamp which is similar to 24869724975. How can I convert this into an epoch timestamp so a I can sort by date in the DB? What I need is to be able to generate a timestamp from the current date and compare to the timestamps stored in the DBs as primary keys

From the README, sonyflake uses 63 bits. This aligns with the design of Snowflake ID, which is specifically designed to fit in a signed integer:

Snowflakes are 64 bits. (Only 63 are used to fit in a signed integer.)

I notice that sonyflake returns uint64 rather than int64:

https://github.com/sony/sonyflake/blob/848d664ceea4c980874f2135c85c42409c530b1f/sonyflake.go#L86

Should it return int64, instead?

I assume not since it has been around for a while. If that's correct, perhaps some documentation could be added to describe how to safely convert the value into an int64 so that it might more easily be used in databases that don't support uint?

Allocating a map[string]uint64 can be very expensive if called from a hot loop. Add a new function DecomposeToBuffer() that writes the ID components into a struct. The new function is implicitly covered by the existing tests because Decompose() uses it under the covers.

I see that it leaves most significant bit unused. That shall make it safe for use with 64 bit signed integer, because it will never go into the negative range from DB perspective (and will stay sortable).

Am I reading it right?

TestSortableID will test if the generated ID is always sortable even when you run in Go routine. So, if you have a system that needs to generate id in application level in a thread, the generated ID can be sorted by those generated ids. Sorted by ID will resemble like sorted by time. And SonyFlake id resembles increment id in the database when you want to sort it.

I'd like to use this code in an AWS Lambda. Is it possible to get a unique 16 bit id for the lambda instance it is running in? The equivalent of AmazonEC2MachineID() but for Lambdas.

As I understand it - Lambda's are limited to 1000 instances - so if we could get something like a "slot" number, ir a unique number from 0 to 999, 1 to 1000 etc - that would be even better and we could reduce the size of the server id field.

@yuokada (or anyone) please let me know if this is possible. I would pay for help with this.

For build systems, unit tests, research, and a range of other contexts it is useful to be able to generate reproducible IDs. Similar to controlling the random seed of a RNG. This branch adds a new method NextReproducibleID() with this feature.

Thanks for an awesome library :-)