Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go

Latest Release Build Status Coverage Status Go ReportCard GoDoc

Concurrency-safe golang caching library with expiration capabilities.

Installation

Make sure you have a working Go environment (Go 1.2 or higher is required). See the install instructions.

To install cache2go, simply run:

go get github.com/muesli/cache2go

To compile it from source:

cd $GOPATH/src/github.com/muesli/cache2go
go get -u -v
go build && go test -v

Example

package main

import (
	"github.com/muesli/cache2go"
	"fmt"
	"time"
)

// Keys & values in cache2go can be of arbitrary types, e.g. a struct.
type myStruct struct {
	text     string
	moreData []byte
}

func main() {
	// Accessing a new cache table for the first time will create it.
	cache := cache2go.Cache("myCache")

	// We will put a new item in the cache. It will expire after
	// not being accessed via Value(key) for more than 5 seconds.
	val := myStruct{"This is a test!", []byte{}}
	cache.Add("someKey", 5*time.Second, &val)

	// Let's retrieve the item from the cache.
	res, err := cache.Value("someKey")
	if err == nil {
		fmt.Println("Found value in cache:", res.Data().(*myStruct).text)
	} else {
		fmt.Println("Error retrieving value from cache:", err)
	}

	// Wait for the item to expire in cache.
	time.Sleep(6 * time.Second)
	res, err = cache.Value("someKey")
	if err != nil {
		fmt.Println("Item is not cached (anymore).")
	}

	// Add another item that never expires.
	cache.Add("someKey", 0, &val)

	// cache2go supports a few handy callbacks and loading mechanisms.
	cache.SetAboutToDeleteItemCallback(func(e *cache2go.CacheItem) {
		fmt.Println("Deleting:", e.Key(), e.Data().(*myStruct).text, e.CreatedOn())
	})

	// Remove the item from the cache.
	cache.Delete("someKey")

	// And wipe the entire cache table.
	cache.Flush()
}

To run this example, go to examples/mycachedapp/ and run:

go run mycachedapp.go

You can find a few more examples here. Also see our test-cases in cache_test.go for further working examples.

Owner
Christian Muehlhaeuser
Geek, Gopher, Software Developer, Maker, Opensource Advocate, Tech Enthusiast, Photographer, Board and Card Gamer
Christian Muehlhaeuser
Comments
  • Question about SetAddedItemCallback

    Question about SetAddedItemCallback

    hello 请看如下函数,是不是应该将这这段注释掉 func (table *CacheTable) SetAddedItemCallback(f func(*CacheItem)) {

    //if len(table.addedItem) > 0 {
    //	table.RemoveAddedItemCallbacks()
    //}
    table.Lock()
    defer table.Unlock()
    table.addedItem = append(table.addedItem, f)
    

    }

  • fix missing dots

    fix missing dots

    Package Testing of go 1.11 is more strict about arguments checking to print-like functions. We will get the error bellow if we left args without dots.

    go test -v
    # github.com/muesli/cache2go
    ./cachetable.go:330: missing ... in args forwarded to print-like function
    FAIL    github.com/muesli/cache2go [build failed]
    

    OS version: manjaro 18.0.0-rc go 1.11.1

  • [BUG] Fatal ERROR when concurrent map write and read

    [BUG] Fatal ERROR when concurrent map write and read

    go version

    go version go1.9 darwin/amd64
    

    log

    fatal error: concurrent map iteration and map write
    
    goroutine 177777 [running]:
    runtime.throw(0xbeb3a9, 0x26)
    	/go1.9/src/runtime/panic.go:605 +0x95 fp=0xc422b05610 sp=0xc422b055f0 pc=0x42e9f5
    runtime.mapiternext(0xc422b05768)
    	/go1.9/src/runtime/hashmap.go:778 +0x6f1 fp=0xc422b056a8 sp=0xc422b05610 pc=0x40d0a1
    github.com/muesli/cache2go.(*CacheTable).expirationCheck(0xc42007e060)
    	/src/github.com/muesli/cache2go/cachetable.go:111 +0x247 fp=0xc422b057d8 sp=0xc422b056a8 pc=0x578d97
    runtime.goexit()
    	/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc422b057e0 sp=0xc422b057d8 pc=0x45f7b1
    created by github.com/muesli/cache2go.(*CacheTable).expirationCheck.func1
    	/src/github.com/muesli/cache2go/cachetable.go:137 +0x3e
    
  • go1.8 can't iterate over map while writing to it

    go1.8 can't iterate over map while writing to it

    I upgrade golang to 1.8 and encounter this panic while using cache2go:

    goroutine 363453 [running]: runtime.throw(0x94121a, 0x26) /usr/local/go/src/runtime/panic.go:596 +0x95 fp=0xc420513de0 sp=0xc420513dc0 runtime.mapiternext(0xc420513f68) /usr/local/go/src/runtime/hashmap.go:737 +0x7ee fp=0xc420513e90 sp=0xc420513de0 github.com/muesli/cache2go.(*CacheTable).expirationCheck(0xc4201d2540) /Users/geomantic/Documents/develop/gopath/src/github.com/muesli/cache2go/cachetable.go:111 +0x336 fp=0xc4205 13fd8 sp=0xc420513e90 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc420513fe0 sp=0xc420513fd8 created by github.com/muesli/cache2go.(*CacheTable).expirationCheck.func1 /Users/geomantic/Documents/develop/gopath/src/github.com/muesli/cache2go/cachetable.go:137 +0x3e

  • why not move table.Lock() into table.addInternal()?

    why not move table.Lock() into table.addInternal()?

    func (table *CacheTable) Add(key interface{}, lifeSpan time.Duration, data interface{}) *CacheItem {
    	item := NewCacheItem(key, lifeSpan, data)
    
    	// Add item to cache.
    	table.Lock()
    	table.addInternal(item)
    
    	return item
    }
    

    In general,Lock() and UnLock() is pair in function,why not move table.Lock() into table.addInternal()?Is there anything other to consider?

  • [Feature Request] Change item lifeSpan

    [Feature Request] Change item lifeSpan

    I'd like to be able to change an item's life span after adding it.
    In my case I want to have it so:

    1. An user interacts with my application
    2. I look for them in the cache, if they're not present I get them from a database.
    3. Make sure they stay in memory while I need them (which can take an arbitrary amount of time)
    4. ... (my program)
    5. After I'm done, set it so they expire in 60 seconds unless the user interacts (which would take us back to the first step).

    Right now this can be done by:

    1. Looking for them in the cache. And if they're not present I get them BUT save it outside the cache.
    2. ... (my program)
    3. After I'm done, If they were in the cache and still are, remove them.
    4. Add them to the cache with a life span of 60 seconds.

    This adds a lot of complexity which would be solved by being able to change the items life span (first to 0 and then to 60).

    Issue implementing it

    I tried implementing it by adding a method to CacheItem (you can see it here), but came across an issue if you:

    1. Create an item with a life span
    2. Set it to 0
    3. Wait until the next expirationCheck (here it stops running because there's no items with lifespan != 0)
    4. Set it to any value different than 0
    5. Now it will never expire because the expirationCheck stopped running

    This issue is the reason the test currently doesn't pass. If I run table.expirationCheck() after resetting the life span it works as expected, but currently there's no way to do it from the method.

  • Panic occurs when you use the new create struct to access the key

    Panic occurs when you use the new create struct to access the key

    Hello, I am a Golang novice. When I was learning your code, I found that I used new() to create the struct. When I access the data, panic prompt Err :nil.Is this a bug?

  • Why creating a Cache requires Double check whether the table exists or not.

    Why creating a Cache requires Double check whether the table exists or not.

    // Cache returns the existing cache table with given name or creates a new one
    // if the table does not exist yet.
    func Cache(table string) *CacheTable {
    	mutex.RLock()
    	t, ok := cache[table]
    	mutex.RUnlock()
    
    	if !ok {
    		mutex.Lock()
    		t, ok = cache[table]
    		// Double check whether the table exists or not.
    		if !ok {
    			t = &CacheTable{
    				name:  table,
    				items: make(map[interface{}]*CacheItem),
    			}
    			cache[table] = t
    		}
    		mutex.Unlock()
    	}
    
    	return t
    }
    
  • table.addedItem does not need to be a array

    table.addedItem does not need to be a array

    if table.addedItem is set, we will remove it before we reset it. so table.addedItem will be always just one item, it didn't need to be a array. table.aboutToDeleteItem is the same.

    `func (table *CacheTable) SetAddedItemCallback(f func(*CacheItem)) {

    // 如果已存在回调函数,则置空
    if len(table.addedItem) > 0 {
    	table.RemoveAddedItemCallbacks()
    }
    table.Lock()
    defer table.Unlock()
    table.addedItem = append(table.addedItem, f)
    

    }`

  • How about start a channel to exec callback

    How about start a channel to exec callback

    Thanks for your open source.

    if the CacheTabls's items becomes very long and the 'expirationCheck' is runing with lots of callback function waiting to execute, it will lock the table for so many time. so I think it' better to make a channle to exec the callback function.

  • question about go use

    question about go use

    In file cache.go

    if !ok { t = &CacheTable{ name: table, items: make(map[interface{}]*CacheItem), }

      mutex.Lock()
      cache[table] = t
      mutex.Unlock()
    

    }

    return t

    the assigment function of cache item is

      t = &CacheTable{ ... }
    

    The "t = &CacheTable" seems to be stored at stack , and can not be reentrant after function returns. How could I understand this ? Thanks.

  • PR to get cached value without updating it's keep alive

    PR to get cached value without updating it's keep alive

    Hi,

    we have a use case where we need to get cached value without touching it's keepAlive value. We need our cached item to expire at whatever time it was about to expire even if we pulled it's value this one time.

    So I added a ValueOnly() function that works the same as CacheTable.Value(), but skips updating of keepAlive. In fact, I moved most of the code of the Value() function to new valueInternal() function and both (Value() and ValueOnly()) use it. It's like addInternal() or deleteInternal() that you also use.

    Test case TestCacheExpire has been extended to cover the new function and all tests pass.

    Please tell what would need to be changed also, to maybe get this additional function merged.

    PR is here: https://github.com/muesli/cache2go/pull/64

  • Adds ValueOnly() function that returns cached value but doesn't update it's keepAlive.

    Adds ValueOnly() function that returns cached value but doesn't update it's keepAlive.

    CacheTable.ValueOnly() works identically to CacheTable.Value() function, but doesn't update it's CacheItem.keepAlive value.

    Can be used to retrieve cached value without touching it or changing any of it's properties.

  • Feature: get the value but not update the expiration time (#57)

    Feature: get the value but not update the expiration time (#57)

    Hi @muesli , As suggested in the Issue https://github.com/muesli/cache2go/issues/57 by you, here is my PR incl. test cases for review.I'm so sorry I only recently noticed your reply on #57 .

  • Feature for the possibility of deletion based on create time.

    Feature for the possibility of deletion based on create time.

    Hi @muesli, As suggested in the Issue #57 by you, here is my PR incl. test cases for review.

    After taking a closer look at the code and now understanding very well how the internal cleanup process works I came to the following solution. By the way, I have to mention that my approach is very similar to #45. I saw it by chance after I had finished my solution. =) It shows that some had the same idea and probably the need for it.

    I am very curious about your feedback

groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache Summary groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many case

Dec 29, 2022
A RESTful caching micro-service in Go backed by Couchbase

Couchcache A caching service developed in Go. It provides REST APIs to access key-value pairs stored in Couchbase. You may also consider using couchca

Sep 26, 2022
Freebase - Proof of concept microservice for A2S INFO message caching
Freebase - Proof of concept microservice for A2S INFO message caching

Freebase A sensible albeit primitive A2S_INFO cache service written in Go. Proof

Feb 23, 2022
Fast thread-safe inmemory cache for big number of entries in Go. Minimizes GC overhead

fastcache - fast thread-safe inmemory cache for big number of entries in Go Features Fast. Performance scales on multi-core CPUs. See benchmark result

Dec 30, 2022
Fast and simple key/value store written using Go's standard library
Fast and simple key/value store written using Go's standard library

Table of Contents Description Usage Cookbook Disadvantages Motivation Benchmarks Test 1 Test 4 Description Package pudge is a fast and simple key/valu

Nov 17, 2022
OcppManager-go - A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables.

?? ocppManager-go A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables. Currently, only

Jan 3, 2022
Eventually consistent distributed in-memory cache Go library

bcache A Go Library to create distributed in-memory cache inside your app. Features LRU cache with configurable maximum keys Eventual Consistency sync

Dec 2, 2022
golang bigcache with clustering as a library.

clusteredBigCache This is a library based on bigcache with some modifications to support clustering and individual item expiration Bigcache is an exce

Sep 26, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Dec 29, 2022
moss - a simple, fast, ordered, persistable, key-val storage library for golang

moss moss provides a simple, fast, persistable, ordered key-val collection implementation as a 100% golang library. moss stands for "memory-oriented s

Dec 18, 2022
Pure Go implementation of D. J. Bernstein's cdb constant database library.

Pure Go implementation of D. J. Bernstein's cdb constant database library.

Oct 19, 2022
A go library for testing Amazon DynamoDB.

minidyn Amazon DynamoDB testing library written in Go. Goals Make local testing for DynamoDB as accurate as possible. Run DynamoDB tests in a CI witho

Nov 9, 2022
redic - Bindings for hiredis Redis-client library

This repo is a fork of https://github.com/redis/hiredis. redic - Bindings for hiredis Redis-client Go library Install go get -u github.com/hjyoun0731/

Dec 21, 2021
Key event handling library for tcell - THIS IS A MIRROR - SEE LINK BELOW

cbind Key event handling library for tcell Features Set KeyEvent handlers Encode and decode KeyEvents as human-readable strings Usage // Create a new

Jan 10, 2022
Dayligo - Golang library for working with Daylio backups

dayligo Dayligo is a library for working with Daylio backup files in the .daylio

Nov 9, 2022
Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on commands and key-regexes written by GO

Welcome to NIPO Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on command

Dec 28, 2022
Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures.

Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures. capabilities which owl provides include Process approval、sql Audit、sql execute and execute as crontab、data backup and recover .

Nov 9, 2022
Being played at The Coffee House and try to find and play it on Spotify
Being played at The Coffee House and try to find and play it on Spotify

The Coffee House Muzik Follow the music that is being played at The Coffee House and try to find and play it on Spotify. Installation Clone this proje

May 25, 2022
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.
Walrus - Fast, Secure and Reliable System Backup, Set up in Minutes.

Walrus is a fast, secure and reliable backup system suitable for modern infrastructure. With walrus, you can backup services like MySQL, PostgreSQL, Redis, etcd or a complete directory with a short interval and low overhead. It supports AWS S3, digitalocean spaces and any S3-compatible object storage service.

Jan 5, 2023