Go client for Redis

Redigo

Build Status GoDoc

Redigo is a Go client for the Redis database.

Features

Documentation

Installation

Install Redigo using the "go get" command:

go get github.com/gomodule/redigo/redis

The Go distribution is Redigo's only dependency.

Related Projects

Contributing

See CONTRIBUTING.md.

License

Redigo is available under the Apache License, Version 2.0.

Comments
  • gomod

    gomod

    Fixes #366 (I think).

    Once merged, additional tags need to be added to the repo:

    • redis/v3.0.0
    • redisx/v3.0.0

    This will make this repo support go modules, and follows the official guidance of bumping the major version when there are released tags that are not actually for go mod.

    It maybe worth considering not having the redis and redisx modules in the same repo. Before taking this change might be a good opportunity to do that since consumers will have to change the import path, anyway.

    At the very least, it may be worthwhile to move the redisx package out of this repo, since it's intended to be experimental and should probably remain v0.

  • what is the 'ERR :0' meaning?

    what is the 'ERR :0' meaning?

    I got a lot of errors , which are 'ERR :0' or 'ERR :1', I got them from my log file. cmds and errors:

    log.2015-01-24.001:2015/01/24 11:26:35 [EE] db.go:59 handleDBQueue(): [asyncDB {233 0xc208f28980 HSET [user:233:accepted_quest:254 step 8]} ERR :0]
    log.2015-01-24.001:2015/01/24 14:07:49 [EE] db.go:59 handleDBQueue(): [asyncDB {301 0xc208386ec0 HSET [user:301:baseInfo shenshou_curr 23]} ERR :0]
    log.2015-01-24.001:2015/01/24 21:13:14 [EE] db.go:59 handleDBQueue(): [asyncDB {557 0xc208fa3a80 HSET [user:557:guides drawcard 999]} ERR :0]
    log.2015-01-25.001:2015/01/25 08:27:21 [EE] db.go:59 handleDBQueue(): [asyncDB {150 0xc208f7ee40 HSET [user:150:baseInfo exp 293]} ERR :0]
    log.2015-01-25.001:2015/01/25 13:10:19 [EE] db.go:59 handleDBQueue(): [asyncDB {63 0xc208880d40 HSET [user:63:guanqiaDetail:69 simpleCanWipe 1]} ERR :1]
    log.2015-01-25.001:2015/01/25 13:25:01 [EE] db.go:59 handleDBQueue(): [asyncDB {688 0xc208d21500 HSET [user:688:guides employ 999]} ERR :1]
    log.2015-01-25.001:2015/01/25 17:37:42 [EE] db.go:59 handleDBQueue(): [asyncDB {752 0xc208f9c700 HSET [user:752:baseInfo shenshou_curr 0]} ERR :1]
    log.2015-01-25.001:2015/01/25 19:31:19 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:133 2927603]} ERR :178]
    log.2015-01-25.001:2015/01/25 19:31:19 [EE] db.go:59 handleDBQueue(): [asyncDB {679 0xc208742c80 HSET [user:679:accepted_quest:5 step 1]} ERR :0]
    log.2015-01-26.001:2015/01/26 02:29:02 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:9 2915959]} ERR :8805]
    log.2015-01-26.001:2015/01/26 02:29:05 [EE] db.go:59 handleDBQueue(): [asyncDB {11 0xc2091b80c0 HSET [user:11:accepted_quest:175 step 4]} ERR :0]
    log.2015-01-26.001:2015/01/26 07:42:15 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:16 8901047]} ERR :415]
    log.2015-01-26.001:2015/01/26 11:30:00 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:108 6737109]} ERR :874]
    log.2015-01-27.001:2015/01/27 01:59:51 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:134 8056353]} ERR :1041]
    log.2015-01-27.001:2015/01/27 09:35:45 [EE] db.go:59 handleDBQueue(): [asyncDB {945 0xc208537280 HSET [user:945:baseInfo exp 318]} ERR :0]
    log.2015-01-27.001:2015/01/27 15:52:24 [EE] db.go:59 handleDBQueue(): [asyncDB {218 0xc20870b400 HSET [user:218:accepted_quest:119 finish 1]} ERR :0]
    log.2015-01-28.001:2015/01/28 00:42:08 [EE] db.go:59 handleDBQueue(): [asyncDB {1034 0xc2086737c0 HSET [user:1034:baseInfo armUpgradeStone 76]} ERR :0]
    log.2015-01-28.001:2015/01/28 02:24:20 [EE] db.go:59 handleDBQueue(): [asyncDB {1070 0xc208838b40 HSET [user:1070:baseInfo login_arm 0]} ERR :0]
    log.2015-01-28.001:2015/01/28 10:51:24 [EE] db.go:59 handleDBQueue(): [asyncDB {205 0xc20d31f140 HSET [user:205:baseInfo power 16]} ERR :0]
    log.2015-01-28.001:2015/01/28 15:06:34 [EE] db.go:59 handleDBQueue(): [asyncDB {614 0xc208838ec0 HSET [user:614:accepted_quest:142 step 6]} ERR :0]
    log.2015-01-28.001:2015/01/28 18:33:58 [EE] db.go:59 handleDBQueue(): [asyncDB {105 0xc208e129c0 SADD [user:105:arms 432]} ERR :1]
    log.2015-01-28.001:2015/01/28 22:06:15 [EE] db.go:59 handleDBQueue(): [asyncDB {1134 0xc208c8d040 HSET [user:1134:guides drawcard 999]} ERR :0]
    log.2015-01-29.001:2015/01/29 14:09:00 [EE] db.go:59 handleDBQueue(): [asyncDB {124 0xc208556b40 HSET [user:124:baseInfo pkCount 5]} ERR :0]
    log.2015-01-29.001:2015/01/29 18:28:22 [EE] db.go:59 handleDBQueue(): [asyncDB {0 <nil> LPUSH [opcode:6 10966403]} ERR :21811]
    

    what does this mean? where does it come from ? It happend sometimes and I couldn't get it in my test.

  • Issue#405 Support for SLOWLOG GET command

    Issue#405 Support for SLOWLOG GET command

    This PR provides support for SLOWLOG GET command results. A new struct SlowLog has been added which contains the details of the SlowLogs obtained by calling the helper SlowLog

    // SlowLog represents a redis SlowLog
    // slowLogID - A unique progressive identifier for every slow log entry.
    // unixTimeStamp - The unix timestamp at which the logged command was processed.
    // executionTime - The amount of time needed for its execution, in microseconds.
    // args - The array composing the arguments of the command.
    // clientIPAddressAndPort - Client IP address and port (4.0 only).
    // clientName - Client name if set via the CLIENT SETNAME command (4.0 only).
    type SlowLog struct {
    	SlowLogID              int64
    	UnixTimeStamp          int64
    	ExecutionTime          int64
    	Args                   []string
    	ClientIPAddressAndPort string
    	ClientName             string
    }
    

    Addresses: #405

  • Support for redis sentinel

    Support for redis sentinel

    Briefly, the SentinelClient supplied by sentinel.go represents a connection to a bundle of sentinels. While only one connection is active at a time, the SentinelClient will failover its internal connection to all configured sentinels before failing any individual operation.

    The SentinelClient has a Dial method, which connects to the sentinel, and DialMaster and DialSlave methods, which connect to the named master or slaves of the named master, respectively.

    The SentinelAwarePool supplied in pool.go is extremely simple. I wanted to avoid an overly-complex implementation here, because I don't have a lot of operational experience with the Pool. The only differences are the addition of a TestOnReturn entry point, which is designed to test returned connections for role changes, and a method to update the internal accounting of the master's address. (Role detection and test wrappers are supplied in sentinel.go). The only meaningful operational difference to the SentinelAwarePool is the ability to purge all idle connections if the master's configuration changes. (active connections will be handled by TestOnReturn).

    An example of usage of the SentinelAwarePool is supplied in pool.go.

    I believe I have supplied sufficient capability here for a barebones but fully capable sentinel configuration, and enough tools and flexibility for a user to build a more complex setup (including using Sentinel pubsub, which is not included here.)

    I've tested this (both pool and standalone connections) on a new 2.8 cluster and an older 2.6-era cluster which does not have CLIENT kill. I would appreciate additional testing and validation if possible, especially on the new 3.0 cluster which I do not have access to.

  • Support Go Modules

    Support Go Modules

    This repo was moved from here from garyburd/redigo. At the time of the move, I made some minor breaking changes to the API and tagged a v2.0.0 release.

    The v2.0.0 release turned out to be a problem when Go Modules were introduced. It's now clear that I should have deleted previous v1.x.x releases from the repo and tagged a new v1.0.0 release.

    I was my intent that this repo have one major semantic version until the next breaking API change is made. Let's call the major semantic version as of today as version N.

    ~~It's possible that users who employ a dependency management tool are using the tags copied over from garyburd/redigo. These users are on major version N-1.~~ Edit: These versions do not compile because they have a package import comment for garyburd/redigo. It's safe to assume that there are no users of these versions.

    Users of Go 1.5+ who do not employ a dependency management tool should continue to get the latest of major version N. This works today and should continue to be supported until Go Modules is widely adopted.

    Users who opt into Go Modules should not need to change their code.

    Because I expect Go Modules to be widely adopted before major version N+1 of this package is created, it's acceptable to require a dependency management tool to get version N+1.

    Adding a go.mod file to a v2.x.x release changes the path of the package to github.com/gomodule/redigo/v2. This requires a change to all importers of the packages and does not work for users on Go 1.8 and earlier.

    See previous discussion in #365 and #363.

  • Feature request:  add support for context.Context

    Feature request: add support for context.Context

    The use of context.Context is recommended for cancellation and deadlines. It would be helpful if redigo could support context.Context and be able to cancel early when needed.

  • New maintainers needed

    New maintainers needed

    I am stepping down as maintainer of of Redigo on May 1st. Use this issue to suggest and discuss new maintainers for the project. I want to have some consensus on the new maintainers before handing the project off.

  • cannot load github.com/gomodule/redigo

    cannot load github.com/gomodule/redigo

    build command-line-arguments: cannot load github.com/gomodule/redigo: module github.com/gomodule/redigo@latest found (v2.0.0+incompatible), but does not contain package github.com/gomodule/redigo

    can anyone help? go version 1.13.11

  • #480 broke Heroku Redis

    #480 broke Heroku Redis

    #480 broke compatibility with Heroku Redis. Before 1.8.2 I was able to connect to Heroku Redis by using the DialURL method. I guess #480 broke this because I started seeing this error after upgrading redigo to 1.8.2: ERR wrong number of arguments for 'auth' command. This is just an assumption but yeah... Will investigate further if I have a bit more time on my hands.

  • Support for context-aware connection

    Support for context-aware connection

    Similar to the ConnWithTimeout interface that implements the Do/Receive methods with an extra time.Duration parameter, I think it would be useful to add in a ConnWithContext interface that takes an extra context.Context parameter. Many other Golang libraries are starting to adopt context.Context usage for cancellation since it was put into the standard library. Additionally, this proposed change shouldn't make any breaking changes to this package's public API. I am happy to implement this feature if I get the green light from whoever maintains this package - I just wanted to start up a discussion first per https://github.com/gomodule/redigo/blob/master/.github/CONTRIBUTING.md.

  • Consider blocking on request for connection from pool when exhausted

    Consider blocking on request for connection from pool when exhausted

    Presently, if a connection from a pool is requested with pool.Get() and the number of connections active has reached MaxActive then Get() will return the error errPoolClosed.

    I feel its more expected by the application developer to always get a connection when calling pool.Get() -- unless the server is down or something. If a function needs to get/set data to redis, then it needs to do that, if the pool is exhausted because of high load on the application, then should it just keep looping asking for a connection until it's not exhausted anymore?

    Perhaps having an async (as it is now) and sync Get() for requesting a connection.. like: pool.Get() to be synchronous and pool.GetOrFail() as async. Perhaps the sync Get() should also return an error after X time of waiting. pool.Get(&redis.Wait{5}) .. and a default config option on the pool struct.

    I wonder how application developers are using redigo's Pool{} stuff, I can imagine most are leaving MaxActive as 0 and just hoping the max number of connections on their system or server never hits. Considering that, there should be a default for MaxActive, not infinity. And MaxIdle should be some reasonable number as well. I wonder how other database drivers do it...? ie. database/sql.

    I think it keeps it much simpler and expected for the app developer. At least for me..

  • panic: runtime error: slice bounds out of range

    panic: runtime error: slice bounds out of range

    Hi, I guess I meet the same concurrency bug as https://github.com/gomodule/redigo/issues/540. This is the panic log:

    panic: runtime error: slice bounds out of range [15:10]
    
    goroutine 16 [running]:
    bufio.(*Reader).ReadSlice(0xc00007b200, 0x8a?)
            /home/work/go_install/go/src/bufio/bufio.go:346 +0x225
    github.com/gomodule/redigo/redis.(*conn).readLine(0xc0004dbac0?)
            /home/work/go/pkg/mod/github.com/gomodule/[email protected]+incompatible/redis/conn.go:431 +0x26
    github.com/gomodule/redigo/redis.(*conn).readReply(0xc00058e8c0)
            /home/work/go/pkg/mod/github.com/gomodule/[email protected]+incompatible/redis/conn.go:504 +0x25
    github.com/gomodule/redigo/redis.(*conn).DoWithTimeout(0xc00058e8c0, 0x0, {0xba10eb, 0x3}, {0xc0005888c0, 0x5, 0x5})
            /home/work/go/pkg/mod/github.com/gomodule/[email protected]+incompatible/redis/conn.go:665 +0x3cf
    github.com/gomodule/redigo/redis.(*conn).Do(0xaee440?, {0xba10eb?, 0xba10eb?}, {0xc0005888c0?, 0x10?, 0xabf780?})
            /home/work/go/pkg/mod/github.com/gomodule/[email protected]+incompatible/redis/conn.go:616 +0x39
    github.com/gomodule/redigo/redis.(*activeConn).Do(0xc000010558, {0xba10eb, 0x3}, {0xc0005888c0, 0x5, 0x5})
            /home/work/go/pkg/mod/github.com/gomodule/[email protected]+incompatible/redis/pool.go:447 +0xea
    

    I want to use redigo to realize a distributed lock.After get the lock, create a new goroutine to update the expire time. Everywhere I use redis, I will get a new client from the redisPool and close it after using. However, I still meet this problem.Here is my code:

    lock

    func (r *RedisLock) Lock() (err error) {
    	for {
    		if err = r.grant(); err != nil {
    			time.Sleep(100 * time.Millisecond)
    		} else {
    			break
    		}
    	}
    	log.Debug("redis key lock success, key: %s", r.key)
    	ctx, cancelFun := context.WithCancel(context.TODO())
    	r.cancelFun = cancelFun
    	r.updateExpire(ctx)
    	r.isLocked = true
    	return nil
    }
    
    func (r *RedisLock) grant() (err error) {
    	grantClient := redisClient.Get()
    	defer grantClient.Close()
    	res, err := redis.String(grantClient.Do("SET", r.key, r.lockId, "EX", r.ttl, "NX"))
    	if res != "OK" {
    		err = errors.New(fmt.Sprintf("grant res is not OK: %s", res))
    		return err
    	}
    
    	return nil
    }
    

    update expire

    func (r *RedisLock) updateExpire(ctx context.Context) {
    	expireClient := redisClient.Get()
    	defer expireClient.Close()
    	go func() {
    		for {
    			select {
    			case <-ctx.Done():
    				return
    			default:
    				res, err := redis.Int(expireClient.Do("EXPIRE", r.key, r.ttl))
    				log.Debug("updateExpire")
    				if res != 1 {
    					if err != nil {
    						log.Error("update redis lock expire error: %s,key : %s", err.Error(), r.key)
    					} else {
    						log.Error("update redis lock expire error,key : %s", r.key)
    					}
    				}
    
    			}
    			time.Sleep(time.Duration(r.ttl/2) * time.Second)
    		}
    	}()
    }
    

    unlock

    func (r *RedisLock) UnLock(lockId string) (err error) {
    	unlockClient := redisClient.Get()
    	defer unlockClient.Close()
    	if r.cancelFun != nil {
    		r.cancelFun()
    	}
    	if r.isLocked {
    		if lockId != r.lockId {
    			err = errors.New(fmt.Sprintf("lockid is different, can't unlock, key: %s", r.key))
    		} else {
    			res, _ := redis.Int(unlockClient.Do("DEL", r.key))
    			if res != 1 {
    				return errors.New(fmt.Sprintf("redis unlock error,key: %s", r.key))
    			}
    		}
    	}
    	return
    }
    

    Thanks !

  • OpenTelemetry tracing manual instrumentation support

    OpenTelemetry tracing manual instrumentation support

    In the same way as seen on the OpenTelemetry instrumentation forgo-redis it would be extremely beneficial to create a similar package for redigo. Not only we can track the overhead of the client code, but we can also precisely trace errors/latency spikes/timeouts/etc...

    Some further info:

    Important Note: Semantic conventions to follow while preparing a package for manual instrumentation.

    While this is not mandatory, we should aim for standardization across clients/languages/DBs. Here's the spec on DB semantic conventions:

    https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/database.md

    PS: will gladly contribute to this matter.

  • add parsing for stream entries

    add parsing for stream entries

    This PR is related to issue https://github.com/gomodule/redigo/issues/375 and adds a first pass at parsing replies containing stream entries into a struct.

    I tested things against a Redis server v5.0.7, which is the first version containing the streams feature: https://redislabs.com/blog/redis-5-0-is-here/.

  • use zero copy improve string and byte convert

    use zero copy improve string and byte convert

    bench result:

    BenchmarkIntForByte-12     34.3          19.8          -42.27%
    
    benchmark                  old allocs     new allocs     delta
    BenchmarkIntForByte-12     1              0              -100.00%
    
    benchmark                  old bytes     new bytes     delta
    BenchmarkIntForByte-12     3             0             -100.00%
    
    
    
    benchmark              old ns/op     new ns/op     delta
    BenchmarkString-12     14.9          2.55          -82.89%
    
    benchmark              old allocs     new allocs     delta
    BenchmarkString-12     1              0              -100.00%
    
    benchmark              old bytes     new bytes     delta
    BenchmarkString-12     4             0             -100.00%
    
Go client for Redis

Redigo Redigo is a Go client for the Redis database. Features A Print-like API with support for all Redis commands. Pipelining, including pipelined tr

Dec 29, 2022
Type-safe Redis client for Golang
Type-safe Redis client for Golang

Redis client for Golang Join Discord to ask questions. Documentation Reference Examples RealWorld example app Ecosystem Redis Mock. Distributed Locks.

Dec 29, 2022
REST based Redis client built on top of Upstash REST API

An HTTP/REST based Redis client built on top of Upstash REST API.

Jul 31, 2022
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

Jan 1, 2023
Query redis with SQL
Query redis with SQL

reqlite reqlite makes it possible to query data in Redis with SQL. Queries are executed client-side with SQLite (not on the redis server). This projec

Dec 23, 2022
Go library that stores data in Redis with SQL-like schema

Go library that stores data in Redis with SQL-like schema. The goal of this library is we can store data in Redis with table form.

Mar 14, 2022
Golang Redis Postgres to-do Project
Golang Redis Postgres to-do Project

Golang Backend Project Problem Statement Build a to-do application with Golang a

Oct 17, 2022
A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.
A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.

A demo project that automatically restarts with a trio of docker, redis and go and transmits page visits.

Feb 6, 2022
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dec 30, 2022
Go client for AMQP 0.9.1

Go RabbitMQ Client Library This is an AMQP 0.9.1 client with RabbitMQ extensions in Go. Project Maturity This project has been used in production syst

Jan 6, 2023
Interactive client for PostgreSQL and MySQL
Interactive client for PostgreSQL and MySQL

dblab Interactive client for PostgreSQL and MySQL. Overview dblab is a fast and lightweight interactive terminal based UI application for PostgreSQL a

Jan 8, 2023
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dec 30, 2022
[mirror] the database client and tools for the Go vulnerability database

The Go Vulnerability Database golang.org/x/vulndb This repository is a prototype of the Go Vulnerability Database. Read the Draft Design. Neither the

Dec 29, 2022
Migration tool for ksqlDB, which uses the ksqldb-go client.
Migration tool for ksqlDB, which uses the ksqldb-go client.

ksqldb-migrate Migration tool for ksqlDB, which uses the ksqldb-go client.

Nov 15, 2022
A client for TiKV

client-tikv ./tikv-client --pd 127.0.0.1:2379,127.0.0.2:2379,127.0.0.3:2379 usage You can query the value directly according to the key. tikv> select

Apr 16, 2022
Client to import measurements to timestream databases.

Timestream DB Client Client to import measurements to timestream databases. Supported Databases/Services AWS Timestream AWS Timestream Run NewTimestre

Jan 11, 2022
Go-clickhouse - ClickHouse client for Go

ClickHouse client for Go 1.18+ This client uses native protocol to communicate w

Jan 9, 2023
Golang client for redislabs' ReJSON module with support for multilple redis clients (redigo, go-redis)

Go-ReJSON - a golang client for ReJSON (a JSON data type for Redis) Go-ReJSON is a Go client for ReJSON Redis Module. ReJSON is a Redis module that im

Dec 25, 2022
Redis client Mock Provide mock test for redis query

Redis client Mock Provide mock test for redis query, Compatible with github.com/go-redis/redis/v8 Install Confirm that you are using redis.Client the

Dec 27, 2022
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.

GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right

Apr 27, 2022