A thread safe map which has expiring key-value pairs

~ timedmap ~

A map which has expiring key-value pairs.

       

go get github.com/zekroTJA/timedmap

Intro

This package allows to set values to a map which will expire and disappear after a specified time.

Here you can read the docs of this package, generated by pkg.go.dev.


Usage Example

package main

import (
	"log"
	"time"

	"github.com/zekroTJA/timedmap"
)

func main() {
	// Create a timed map with a cleanup timer interval of 1 second
	tm := timedmap.New(1 * time.Second)
	// Set value of key "hey" to 213, which will expire after 3 seconds
	tm.Set("hey", 213, 3*time.Second)
	// Print the value of "hey"
	printKeyVal(tm, "hey")
	// Block the main thread for 5 seconds
	// After this time, the key-value pair "hey": 213 has expired
	time.Sleep(5 * time.Second)
	// Now, this function should show that there is no key "hey"
	// in the map, because it has been expired
	printKeyVal(tm, "hey")
}

func printKeyVal(tm *timedmap.TimedMap, key interface{}) {
	d, ok := tm.GetValue(key).(int)
	if !ok {
		log.Println("data expired")
		return
	}

	log.Printf("%v = %d\n", key, d)
}

Further examples, you can find in the example directory.

If you want to see this package in a practcal use case scenario, please take a look at the rate limiter implementation of the REST API of myrunes.com, where I have used timedmap for storing client-based limiter instances:
https://github.com/myrunes/backend/blob/master/internal/ratelimit/ratelimit.go


Copyright (c) 2020 zekro Development (Ringo Hoffmann).
Covered by MIT licence.

Owner
zekro
Young, learning, full-stack developer. I <3 Go, TypeScript and C#. Playing a lot with web stuff.
zekro
Similar Resources

Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

mxj - to/from maps, XML and JSON Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-

Dec 22, 2022

A typed implementation of the Go sync.Map using code generation

syncmap A typed implementation of the Go sync.Map using code generation. Install go get -u github.com/a8m/syncmap@master Examples: Using CLI $ syncma

Dec 26, 2022

An in-memory string-interface{} map with various expiration options for golang

TTLCache - an in-memory cache with expiration TTLCache is a simple key/value cache in golang with the following functions: Expiration of items based o

Dec 28, 2022

A fast (5x) string keyed read-only map for Go - particularly good for keys using a small set of nearby runes.

faststringmap faststringmap is a fast read-only string keyed map for Go (golang). For our use case it is approximately 5 times faster than using Go's

Jan 8, 2023

💯 Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving

Package validator implements value validations for structs and individual fields based on tags.

Nov 9, 2022

A Go library to iterate over potentially nested map keys using the visitor pattern

A Go library to iterate over potentially nested map keys using the visitor pattern

Mar 15, 2022

Go library for encoding native Go structures into generic map values.

wstructs origin: github.com/things-go/structs Go library for encoding native Go structures into generic map values. Installation Use go get. go ge

Jan 10, 2022

Type-safe, zero-allocation sets for Go

Set Package set is a type-safe, zero-allocation port of the excellent package fatih/set. It contains sets for most of the basic types and you can gene

Jan 5, 2023

dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

Dec 19, 2022
Comments
  • fatal error: concurrent map writes

    fatal error: concurrent map writes

    
    fatal error: concurrent map writes
    
    goroutine 53 [running]:
    runtime.throw(0x21d9a37, 0x15)
            /usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc00059a5a8 sp=0xc00059a578 pc=0x56b952
    runtime.mapdelete(0x204f0a0, 0xc00032e180, 0xc00059a638)
            /usr/local/go/src/runtime/map.go:702 +0x46e fp=0xc00059a608 sp=0xc00059a5a8 pc=0x544c6e
    github.com/zekroTJA/timedmap.(*TimedMap).expireElement(0xc000618040, 0x2000940, 0xc007e1d690, 0x0, 0xc006694300)
            /root/go/pkg/mod/github.com/zekro!t!j!a/[email protected]/timedmap.go:177 +0xc5 fp=0xc00059a660 sp=0xc00059a608 pc=0x10756e5
    github.com/zekroTJA/timedmap.(*TimedMap).cleanUp(0xc000618040)
            /root/go/pkg/mod/github.com/zekro!t!j!a/[email protected]/timedmap.go:190 +0x1a9 fp=0xc00059a758 sp=0xc00059a660 pc=0x10758a9
    github.com/zekroTJA/timedmap.New.func1(0xc0000cc3c0, 0xc000618040)
            /root/go/pkg/mod/github.com/zekro!t!j!a/[email protected]/timedmap.go:62 +0x97 fp=0xc00059a7d0 sp=0xc00059a758 pc=0x1075f77
    runtime.goexit()
            /usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00059a7d8 sp=0xc00059a7d0 pc=0x5a1ce1
    created by github.com/zekroTJA/timedmap.New
            /root/go/pkg/mod/github.com/zekro!t!j!a/[email protected]/timedmap.go:58 +0xc5
    
    
  • Using sync.Pool for creating element values

    Using sync.Pool for creating element values

    Currently, each time a new element is set as value to the map, a new element object is allocated on the heap and the pointer of it then set as value to the map.

    This causes a lot of allocations when setting new key-value pairs. This might be bypassed by using a sync.Pool to create and cache element objects so they can be re-used after a value was deleted from the map, which often happens when values are expired.

    See this post as "template" on how to implement and use sync.Pool: https://www.akshaydeo.com/blog/2017/12/23/How-did-I-improve-latency-by-700-percent-using-syncPool

  • Sections implementation rework

    Sections implementation rework

    Currently, sections are implemented as additional part of the key object to seperate one single main map into different sections.

    So, everytime, an object from the map is accessed, a keyWrap object needs to be created on the stack to select from the map with it as key. This increases the ammount of allocations per access to the map.

    This can be bypassed by creating a completely seperate map for each section and access to the map only happens by the key interface objects. The sections store can be someting like a map[int]map[interface{}]*element like type, maybe wrapped with an Section struct like object.

Package ring provides a high performance and thread safe Go implementation of a bloom filter.

ring - high performance bloom filter Package ring provides a high performance and thread safe Go implementation of a bloom filter. Usage Please see th

Nov 20, 2022
go.fifo provides a simple fifo thread-safe queue for the Go programming language

go.fifo Description go.fifo provides a simple FIFO thread-safe queue. *fifo.Queue supports pushing an item at the end with Add(), and popping an item

Aug 29, 2022
A Golang lock-free thread-safe HashMap optimized for fastest read access.

hashmap Overview A Golang lock-free thread-safe HashMap optimized for fastest read access. Usage Set a value for a key in the map: m := &HashMap{} m.S

Dec 30, 2022
A simple and efficient thread-safe sharded hashmap for Go

shardmap A simple and efficient thread-safe sharded hashmap for Go. This is an alternative to the standard Go map and sync.Map, and is optimized for w

Dec 17, 2022
skipmap is a high-performance concurrent sorted map based on skip list. Up to 3x ~ 10x faster than sync.Map in the typical pattern.
skipmap is a high-performance concurrent sorted map based on skip list. Up to 3x ~ 10x faster than sync.Map in the typical pattern.

Introduction skipmap is a high-performance concurrent map based on skip list. In typical pattern(one million operations, 90%LOAD 9%STORE 1%DELETE), th

Jan 8, 2023
Recursively searches a map[string]interface{} structure for another map[string]interface{} structure

msirecurse Recursively searches a map[string]interface{} structure for existence of a map[string]interface{} structure Motivation I wrote this package

Mar 3, 2022
Null Types, Safe primitive type conversion and fetching value from complex structures.

Typ Typ is a library providing a powerful interface to impressive user experience with conversion and fetching data from built-in types in Golang Feat

Sep 26, 2022
Fast in-memory key:value store/cache with TTL

MCache library go-mcache - this is a fast key:value storage. Its major advantage is that, being essentially a thread-safe . map[string]interface{} wit

Nov 11, 2022
An in-memory string-interface{} map with various expiration options for golang

TTLCache - an in-memory cache with expiration TTLCache is a simple key/value cache in golang with the following functions: Expiration of items based o

Jan 8, 2023
A prefix-enhanced map in Go

PrefixMap PrefixMap is a prefix-enhanced map that eases the retrieval of values based on key prefixes. Quick Start Creating a PrefixMap // creates the

Jun 13, 2022