A typed implementation of the Go sync.Map using code generation

syncmap

https://godoc.org/github.com/a8m/syncmap LICENSE Build Status Go Report Card

A typed implementation of the Go sync.Map using code generation.

Install

go get -u github.com/a8m/syncmap@master

Examples:

  1. Using CLI
$ syncmap -name IntMap "map[int]int"
$ syncmap -name RequestMap -pkg mypkg "map[string]*http.Request"

Or:

$ go run github.com/a8m/syncmap -name IntMap "map[int]int"

  1. Using go generate.

    • Add a directive with map definition: 
      //go:generate go run github.com/a8m/syncmap -name WriterMap map[string]io.Writer

//go:generate go run github.com/a8m/syncmap -name Requests map[string]*http.Request
    • Then, run go generate on this package.

    See testdata/gen.go for more examples.

How does it work?

syncmap didn't copy the code of sync/map.go and replace its identifiers. Instead, it reads the sync/map.go from your GOROOT, parses it into an *ast.File, and runs a few mutators that bring it to the desired state. Check the code for more information.

How can we make sure it will continue to work? - I'm running a daily CI test on TravisCI.

Benchmark

Benchmark tests were taken from the sync package.

BenchmarkLoadMostlyHits/*main.DeepCopyMap-8         	100000000	        15.1 ns/op
BenchmarkLoadMostlyHits/*main.RWMutexMap-8          	30000000	        54.4 ns/op
BenchmarkLoadMostlyHits/*sync.Map-8                 	100000000	        14.0 ns/op
BenchmarkLoadMostlyHits/*main.IntMap-8              	300000000	        5.65 ns/op <--

BenchmarkLoadMostlyMisses/*main.DeepCopyMap-8       	200000000	        10.2 ns/op
BenchmarkLoadMostlyMisses/*main.RWMutexMap-8        	30000000	        59.2 ns/op
BenchmarkLoadMostlyMisses/*sync.Map-8               	100000000	        11.3 ns/op
BenchmarkLoadMostlyMisses/*main.IntMap-8            	300000000	        4.05 ns/op <--

BenchmarkLoadOrStoreBalanced/*main.RWMutexMap-8     	 3000000	        400 ns/op
BenchmarkLoadOrStoreBalanced/*sync.Map-8            	 3000000	        400 ns/op
BenchmarkLoadOrStoreBalanced/*main.IntMap-8         	 5000000	        233 ns/op <--

BenchmarkLoadOrStoreUnique/*main.RWMutexMap-8       	 2000000	        744 ns/op
BenchmarkLoadOrStoreUnique/*sync.Map-8              	 2000000	        903 ns/op
BenchmarkLoadOrStoreUnique/*main.IntMap-8           	 3000000	        388 ns/op <--

BenchmarkLoadOrStoreCollision/*main.DeepCopyMap-8   	200000000	        7.29 ns/op
BenchmarkLoadOrStoreCollision/*main.RWMutexMap-8    	20000000	        97.5 ns/op
BenchmarkLoadOrStoreCollision/*sync.Map-8           	200000000	        9.11 ns/op
BenchmarkLoadOrStoreCollision/*main.IntMap-8        	500000000	        3.14 ns/op <--

BenchmarkRange/*main.DeepCopyMap-8                  	  500000	        4479 ns/op
BenchmarkRange/*main.RWMutexMap-8                   	   30000	        56834 ns/op
BenchmarkRange/*sync.Map-8                          	  300000	        4464 ns/op
BenchmarkRange/*main.IntMap-8                       	1000000000	        2.38 ns/op <--

BenchmarkAdversarialAlloc/*main.DeepCopyMap-8       	 2000000	        826 ns/op
BenchmarkAdversarialAlloc/*main.RWMutexMap-8        	20000000	        73.6 ns/op
BenchmarkAdversarialAlloc/*sync.Map-8               	 5000000	        303 ns/op
BenchmarkAdversarialAlloc/*main.IntMap-8            	10000000	        182 ns/op <--

BenchmarkAdversarialDelete/*main.DeepCopyMap-8      	10000000	        204 ns/op
BenchmarkAdversarialDelete/*main.RWMutexMap-8       	20000000	        78.3 ns/op
BenchmarkAdversarialDelete/*sync.Map-8              	20000000	        72.2 ns/op
BenchmarkAdversarialDelete/*main.IntMap-8           	100000000	        14.2 ns/op <--

Running benchmark with -benchmem

BenchmarkLoadMostlyHits/*main.DeepCopyMap-8         100000000	  12.7 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyHits/*main.RWMutexMap-8          30000000	  53.6 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyHits/*sync.Map-8                 100000000	  16.3 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyHits/*main.IntMap-8              200000000	  6.02 ns/op	  0 B/op	  0 allocs/op <--

BenchmarkLoadMostlyMisses/*main.DeepCopyMap-8       200000000	  7.99 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyMisses/*main.RWMutexMap-8        30000000	  52.6 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyMisses/*sync.Map-8               200000000	  8.87 ns/op	  7 B/op	  0 allocs/op
BenchmarkLoadMostlyMisses/*main.IntMap-8            1000000000	  2.88 ns/op	  0 B/op	  0 allocs/op <--

BenchmarkLoadOrStoreBalanced/*main.RWMutexMap-8     3000000	  357 ns/op	  71 B/op	  2 allocs/op
BenchmarkLoadOrStoreBalanced/*sync.Map-8            3000000	  417 ns/op	  70 B/op	  3 allocs/op
BenchmarkLoadOrStoreBalanced/*main.IntMap-8         5000000	  202 ns/op	  42 B/op	  1 allocs/op <--

BenchmarkLoadOrStoreUnique/*main.RWMutexMap-8       2000000	  648 ns/op	  178 B/op	  2 allocs/op
BenchmarkLoadOrStoreUnique/*sync.Map-8              2000000	  745 ns/op	  163 B/op	  4 allocs/op
BenchmarkLoadOrStoreUnique/*main.IntMap-8           3000000	  368 ns/op	  74 B/op	  2 allocs/op <--

BenchmarkLoadOrStoreCollision/*main.DeepCopyMap-8   300000000	  5.90 ns/op	  0 B/op	  0 allocs/op
BenchmarkLoadOrStoreCollision/*main.RWMutexMap-8    20000000	  94.5 ns/op	  0 B/op	  0 allocs/op
BenchmarkLoadOrStoreCollision/*sync.Map-8           200000000	  7.55 ns/op	  0 B/op	  0 allocs/op
BenchmarkLoadOrStoreCollision/*main.IntMap-8        1000000000	  2.68 ns/op	  0 B/op	  0 allocs/op <--

BenchmarkRange/*main.DeepCopyMap-8                  500000	  3376 ns/op	  0 B/op	  0 allocs/op
BenchmarkRange/*main.RWMutexMap-8                   30000	  56675 ns/op	  16384 B/op	  1 allocs/op
BenchmarkRange/*sync.Map-8                          500000	  3587 ns/op	  0 B/op	  0 allocs/op
BenchmarkRange/*main.IntMap-8                       2000000000	  1.75 ns/op	  0 B/op	  0 allocs/op <--

BenchmarkAdversarialAlloc/*main.DeepCopyMap-8       2000000	  761 ns/op	  535 B/op	  1 allocs/op
BenchmarkAdversarialAlloc/*main.RWMutexMap-8        20000000	  67.9 ns/op	  8 B/op	  1 allocs/op
BenchmarkAdversarialAlloc/*sync.Map-8               5000000	  264 ns/op	  51 B/op	  1 allocs/op
BenchmarkAdversarialAlloc/*main.IntMap-8            10000000	  176 ns/op	  28 B/op	  0 allocs/op <--

BenchmarkAdversarialDelete/*main.DeepCopyMap-8      10000000	  194 ns/op	  168 B/op	  1 allocs/op
BenchmarkAdversarialDelete/*main.RWMutexMap-8       20000000	  76.9 ns/op	  25 B/op	  1 allocs/op
BenchmarkAdversarialDelete/*sync.Map-8              20000000	  60.8 ns/op	  18 B/op	  1 allocs/op
BenchmarkAdversarialDelete/*main.IntMap-8           100000000	  13.1 ns/op	  0 B/op	  0 allocs/op <--

LICENSE

I am providing code in the repository to you under MIT license. Because this is my personal repository, the license you receive to my code is from me and not my employer (Facebook)

Owner
Ariel Mashraki
Ariel Mashraki
https://github.com/a8m/syncmap
Comments
  • syncmap: function was deleted

    syncmap: function was deleted

    Hi @a8m ,

    No matter which example command you mentioned in the doc' I'm always ending with this error:

    sneko@sneko XXXXX % go run github.com/a8m/syncmap -name IntMap "map[int]int"
syncmap: function was deleted

    I also tried directly yncmap -name IntMap "map[int]int" but same error.

    Any idea of what I'm missing that triggers (?) https://github.com/a8m/syncmap/blob/4bbbd178de97432c071e2d70312b77ec0039fefe/syncmap.go#L128

    Thank you,

  • not work on go 1.18 any more

    not work on go 1.18 any more

    the simple cli: syncmap -name TsysLicMap -pkg mypkg "map[string]*http.Request" generate the code type is any.

    func (m *TsysLicMap) Load(key any) (value any, ok bool)
  • Maybe you should copy a syncmap.go as a template, not user's GOROOT

    Maybe you should copy a syncmap.go as a template, not user's GOROOT

    Different versions of syncmap.go are different, including unexport methods, fields, variables and so on. Users may fail in some old go-version. So you can use a template to keep the program stable and update with the go version.

  • The case BenchmarkRange seems has some problem ?

    The case BenchmarkRange seems has some problem ? 

    // syncmap code:
	b.Run(fmt.Sprintf("%T", &IntMap{}), func(b *testing.B) {
		m := &IntMap{}
		// setup:

		// reset:
		b.ResetTimer()

		// perG:
		perG := func(b *testing.B, pb *testing.PB, i int, m *IntMap) {
			for ; pb.Next(); i++ {
				m.Range(func(_, _ int) bool { return true })
			}
		}
		var i int64
		b.RunParallel(func(pb *testing.PB) {
			id := int(atomic.AddInt64(&i, 1) - 1)
			perG(b, pb, id*b.N, m)
		})
	})

    seems the setup function is nil, then the map is empy, so at the benrchmark result, it seems vary fast

  • Can different custom packages for key and value is possible?

    Can different custom packages for key and value is possible?

    Hi @a8m ,

    Currently using a basic

    syncmap -name UserChannelsMap -pkg xmap -o ./types/xmap/user_channels_map.go "map[string]string"

    will work.

    If I want to use a custom type:

    syncmap -name UserChannelsMap -pkg xmap -o ./types/xmap/user_channels_map.go "map[string]*UserChannelList"

    I need to make sure UserChannelList is in the same package than the one exported for the file (here xmap). I could arrange myself to do so, but it brings some complexity.

    Last case, I want to use like an "enum" for the keys, in my case it's defined in my own constant package, how could I deal with this? From what I understand it's not possible, the syncmap would have to manage some add of imports, no?

    syncmap -name UserChannelsMap -pkg xmap -o ./types/xmap/user_channels_map.go "map[constant.NameID]*UserChannelList"

    Maybe I missed something 🤔

    Thank you,

    EDIT: a workaround could be that aside my exported file (having package xmap) I could declare type aliases for both the key and the value types, like:

    import ...

type NameID = constant.NameID
type UserChannelList = entity.UserChannelList

    EDIT2: in all cases, I should make sure it won't bring circular loop if the XXXMap was needed in one of those packages ^^

Related tags
Data Structures go map sync generics code-generation concurrent-map syncmap
Recursively searches a map[string]interface{} structure for another map[string]interface{} structure

msirecurse Recursively searches a map[string]interface{} structure for existence of a map[string]interface{} structure Motivation I wrote this package

Mar 3, 2022
Provides conversion from athena outputs to strongly-typed data models.
 Provides conversion from athena outputs to strongly-typed data models.

Provides conversion from athena outputs to strongly defined data models. Getting started Given the following data struct you define: type MyModel stru

Feb 7, 2022
Sync distributed sets using bloom filters

goSetReconciliation An implementation to sync distributed sets using bloom filters. Based on the paper "Low complexity set reconciliation using Bloom

Jan 4, 2022
A parser generator where rules defined as go structs and code generation is optional

A parser generator where rules defined as go structs and code generation is optional. The concepts are introduced in the simple example below.

Jul 1, 2022
A fast (5x) string keyed read-only map for Go - particularly good for keys using a small set of nearby runes.

faststringmap faststringmap is a fast read-only string keyed map for Go (golang). For our use case it is approximately 5 times faster than using Go's

Jan 8, 2023
A Go library to iterate over potentially nested map keys using the visitor pattern

A Go library to iterate over potentially nested map keys using the visitor pattern

Mar 15, 2022
sync.WaitGroup for concurrent use

concwg Description This package provides a version of sync.WaitGroup that allows calling Add and Wait in different goroutines. Motivation sync.WaitGro

May 20, 2022
A thread safe map which has expiring key-value pairs

~ timedmap ~ A map which has expiring key-value pairs. go get github.com/zekroTJA/timedmap Intro This package allows to set values to a map which will

Dec 29, 2022
An in-memory string-interface{} map with various expiration options for golang

TTLCache - an in-memory cache with expiration TTLCache is a simple key/value cache in golang with the following functions: Expiration of items based o

Jan 8, 2023
A prefix-enhanced map in Go

PrefixMap PrefixMap is a prefix-enhanced map that eases the retrieval of values based on key prefixes. Quick Start Creating a PrefixMap // creates the

Jun 13, 2022
Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

mxj - to/from maps, XML and JSON Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-

Dec 22, 2022
An in-memory string-interface{} map with various expiration options for golang

TTLCache - an in-memory cache with expiration TTLCache is a simple key/value cache in golang with the following functions: Expiration of items based o

Dec 28, 2022
💯 Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving

Package validator implements value validations for structs and individual fields based on tags.

Nov 9, 2022
Go library for encoding native Go structures into generic map values.

wstructs origin: github.com/things-go/structs Go library for encoding native Go structures into generic map values. Installation Use go get. go ge

Jan 10, 2022
Leet code - Practice code of leetcode

leet_code Practice code of leetcode. Setting : "go.gopath": "/Users//go", path:

Jan 6, 2022
A simple Set data structure implementation in Go (Golang) using LinkedHashMap.

Set Set is a simple Set data structure implementation in Go (Golang) using LinkedHashMap. This library allow you to get a set of int64 or string witho

Sep 26, 2022
Simple code just to try out and Binary Tree on Golang.

Character counter | ▮▮▮▮▮▮▮▮ Simple code just to try out and Binary Tree on Golang. Count characters to train openning a file and reading it, as well

May 17, 2022
LeetCode in Go with the code style strictly follows the Google Golang Style Guide
 LeetCode in Go with the code style strictly follows the Google Golang Style Guide

LeetCode in Go LeetCode Online Judge is a website containing many algorithm questions. Most of them are real interview questions of Google, Facebook,

Nov 13, 2021
Tutorial code for my video Beginner Programming with Golang - Conways Game of Life

Beginner Programming with Golang - Conways Game of Life Who is this tutorial for? Programming beginners and people with programming experience who wan

Jan 26, 2022