Redcon is a custom Redis server framework for Go that is fast and simple to use.

REDCON
GoDoc

Fast Redis compatible server framework for Go

Redcon is a custom Redis server framework for Go that is fast and simple to use. The reason for this library it to give an efficient server front-end for the BuntDB and Tile38 projects.

Features

  • Create a Fast custom Redis compatible server in Go
  • Simple interface. One function ListenAndServe and two types Conn & Command
  • Support for pipelining and telnet commands
  • Works with Redis clients such as redigo, redis-py, node_redis, and jedis
  • TLS Support
  • Compatible pub/sub support
  • Multithreaded

Installing

go get -u github.com/tidwall/redcon

Example

Here's a full example of a Redis clone that accepts:

  • SET key value
  • GET key
  • DEL key
  • PING
  • QUIT
  • PUBLISH channel message
  • SUBSCRIBE channel

You can run this example from a terminal:

go run example/clone.go
package main

import (
	"log"
	"strings"
	"sync"

	"github.com/tidwall/redcon"
)

var addr = ":6380"

func main() {
	var mu sync.RWMutex
	var items = make(map[string][]byte)
	var ps redcon.PubSub
	go log.Printf("started server at %s", addr)
	err := redcon.ListenAndServe(addr,
		func(conn redcon.Conn, cmd redcon.Command) {
			switch strings.ToLower(string(cmd.Args[0])) {
			default:
				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
			case "ping":
				conn.WriteString("PONG")
			case "quit":
				conn.WriteString("OK")
				conn.Close()
			case "set":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				items[string(cmd.Args[1])] = cmd.Args[2]
				mu.Unlock()
				conn.WriteString("OK")
			case "get":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.RLock()
				val, ok := items[string(cmd.Args[1])]
				mu.RUnlock()
				if !ok {
					conn.WriteNull()
				} else {
					conn.WriteBulk(val)
				}
			case "del":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				_, ok := items[string(cmd.Args[1])]
				delete(items, string(cmd.Args[1]))
				mu.Unlock()
				if !ok {
					conn.WriteInt(0)
				} else {
					conn.WriteInt(1)
				}
			case "publish":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				conn.WriteInt(ps.Publish(string(cmd.Args[1]), string(cmd.Args[2])))
			case "subscribe", "psubscribe":
				if len(cmd.Args) < 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				command := strings.ToLower(string(cmd.Args[0]))
				for i := 1; i < len(cmd.Args); i++ {
					if command == "psubscribe" {
						ps.Psubscribe(conn, string(cmd.Args[i]))
					} else {
						ps.Subscribe(conn, string(cmd.Args[i]))
					}
				}
			}
		},
		func(conn redcon.Conn) bool {
			// Use this function to accept or deny the connection.
			// log.Printf("accept: %s", conn.RemoteAddr())
			return true
		},
		func(conn redcon.Conn, err error) {
			// This is called when the connection has been closed
			// log.Printf("closed: %s, err: %v", conn.RemoteAddr(), err)
		},
	)
	if err != nil {
		log.Fatal(err)
	}
}

TLS Example

Redcon has full TLS support through the ListenAndServeTLS function.

The same example is also provided for serving Redcon over TLS.

go run example/tls/clone.go

Benchmarks

Redis: Single-threaded, no disk persistence.

$ redis-server --port 6379 --appendonly no
redis-benchmark -p 6379 -t set,get -n 10000000 -q -P 512 -c 512
SET: 941265.12 requests per second
GET: 1189909.50 requests per second

Redcon: Single-threaded, no disk persistence.

$ GOMAXPROCS=1 go run example/clone.go
redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 2018570.88 requests per second
GET: 2403846.25 requests per second

Redcon: Multi-threaded, no disk persistence.

$ GOMAXPROCS=0 go run example/clone.go
$ redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 1944390.38 requests per second
GET: 3993610.25 requests per second

Running on a MacBook Pro 15" 2.8 GHz Intel Core i7 using Go 1.7

Contact

Josh Baker @tidwall

License

Redcon source code is available under the MIT License.

Comments
  • Latency is high

    Latency is high

    Throughput of Redcon seems really good but latency is really high in comparison to Redis with shallow or deep pipelines. Even if you make the Redis commands NOOP.

    I created a SET command that only wrote a response to benchmark the I/O performance of redcon versus Redis. Keep in mind Redis is doing key mutations while my benchmark is not.

    conn.WriteString("OK")
    

    Redis

    redis-benchmark -h server -p 6379 -t set -n 10000000 -P 1 -c 768
    ====== SET ======
      10000000 requests completed in 101.57 seconds
      768 parallel clients
      3 bytes payload
      keep alive: 1
    
    0.00% <= 2 milliseconds
    0.00% <= 3 milliseconds
    76.97% <= 4 milliseconds
    99.98% <= 5 milliseconds
    99.99% <= 6 milliseconds
    100.00% <= 7 milliseconds
    100.00% <= 8 milliseconds
    100.00% <= 8 milliseconds
    98455.23 requests per second
    
    redis-benchmark -h server -p 6379 -t set -n 10000000 -P 256 -c 768
    ====== SET ======
      10000000 requests completed in 8.61 seconds
      768 parallel clients
      3 bytes payload
      keep alive: 1
    
    0.00% <= 8 milliseconds
    0.00% <= 9 milliseconds
    0.03% <= 10 milliseconds
    0.06% <= 78 milliseconds
    0.06% <= 79 milliseconds
    0.10% <= 80 milliseconds
    0.13% <= 81 milliseconds
    0.14% <= 82 milliseconds
    0.16% <= 84 milliseconds
    0.17% <= 85 milliseconds
    0.18% <= 86 milliseconds
    0.19% <= 87 milliseconds
    0.22% <= 88 milliseconds
    0.27% <= 89 milliseconds
    0.32% <= 90 milliseconds
    0.35% <= 91 milliseconds
    0.36% <= 92 milliseconds
    0.36% <= 96 milliseconds
    0.38% <= 97 milliseconds
    0.39% <= 98 milliseconds
    0.41% <= 99 milliseconds
    0.43% <= 100 milliseconds
    0.48% <= 101 milliseconds
    0.55% <= 102 milliseconds
    0.62% <= 103 milliseconds
    0.68% <= 104 milliseconds
    0.75% <= 105 milliseconds
    0.83% <= 106 milliseconds
    0.91% <= 107 milliseconds
    0.98% <= 108 milliseconds
    1.07% <= 109 milliseconds
    1.16% <= 110 milliseconds
    1.23% <= 111 milliseconds
    1.29% <= 112 milliseconds
    1.35% <= 113 milliseconds
    1.42% <= 114 milliseconds
    1.47% <= 115 milliseconds
    1.54% <= 116 milliseconds
    1.60% <= 117 milliseconds
    1.66% <= 118 milliseconds
    1.73% <= 119 milliseconds
    1.80% <= 120 milliseconds
    1.86% <= 121 milliseconds
    1.89% <= 122 milliseconds
    1.91% <= 128 milliseconds
    1.94% <= 129 milliseconds
    2.05% <= 130 milliseconds
    2.24% <= 131 milliseconds
    2.57% <= 132 milliseconds
    3.32% <= 133 milliseconds
    4.32% <= 134 milliseconds
    5.45% <= 135 milliseconds
    6.56% <= 136 milliseconds
    7.74% <= 137 milliseconds
    9.23% <= 138 milliseconds
    10.92% <= 139 milliseconds
    12.76% <= 140 milliseconds
    14.70% <= 141 milliseconds
    16.73% <= 142 milliseconds
    18.77% <= 143 milliseconds
    20.73% <= 144 milliseconds
    22.69% <= 145 milliseconds
    24.62% <= 146 milliseconds
    26.42% <= 147 milliseconds
    27.84% <= 148 milliseconds
    29.06% <= 149 milliseconds
    30.14% <= 150 milliseconds
    30.95% <= 151 milliseconds
    31.64% <= 152 milliseconds
    32.47% <= 153 milliseconds
    33.75% <= 154 milliseconds
    35.64% <= 155 milliseconds
    38.12% <= 156 milliseconds
    41.20% <= 157 milliseconds
    44.65% <= 158 milliseconds
    48.27% <= 159 milliseconds
    52.01% <= 160 milliseconds
    55.82% <= 161 milliseconds
    59.59% <= 162 milliseconds
    63.34% <= 163 milliseconds
    67.20% <= 164 milliseconds
    71.02% <= 165 milliseconds
    74.71% <= 166 milliseconds
    77.64% <= 167 milliseconds
    79.56% <= 168 milliseconds
    81.13% <= 169 milliseconds
    82.53% <= 170 milliseconds
    83.86% <= 171 milliseconds
    85.15% <= 172 milliseconds
    86.44% <= 173 milliseconds
    87.65% <= 174 milliseconds
    88.88% <= 175 milliseconds
    90.05% <= 176 milliseconds
    91.20% <= 177 milliseconds
    92.25% <= 178 milliseconds
    93.18% <= 179 milliseconds
    94.07% <= 180 milliseconds
    94.89% <= 181 milliseconds
    95.64% <= 182 milliseconds
    96.34% <= 183 milliseconds
    96.90% <= 184 milliseconds
    97.33% <= 185 milliseconds
    97.63% <= 186 milliseconds
    97.90% <= 187 milliseconds
    98.12% <= 188 milliseconds
    98.36% <= 189 milliseconds
    98.58% <= 190 milliseconds
    98.79% <= 191 milliseconds
    98.98% <= 192 milliseconds
    99.16% <= 193 milliseconds
    99.34% <= 194 milliseconds
    99.48% <= 195 milliseconds
    99.60% <= 196 milliseconds
    99.71% <= 197 milliseconds
    99.84% <= 198 milliseconds
    99.91% <= 199 milliseconds
    99.96% <= 200 milliseconds
    99.99% <= 201 milliseconds
    100.00% <= 201 milliseconds
    1161170.38 requests per second
    

    Redcon

    redis-benchmark -h server -p 6380 -t set -n 10000000 -P 1 -c 768
    ====== SET ======
      10000000 requests completed in 90.21 seconds
      768 parallel clients
      3 bytes payload
      keep alive: 1
    
    0.01% <= 2 milliseconds
    28.35% <= 3 milliseconds
    81.28% <= 4 milliseconds
    99.97% <= 5 milliseconds
    99.98% <= 6 milliseconds
    99.98% <= 260 milliseconds
    99.98% <= 261 milliseconds
    99.98% <= 262 milliseconds
    99.98% <= 263 milliseconds
    99.98% <= 264 milliseconds
    99.98% <= 265 milliseconds
    99.98% <= 266 milliseconds
    99.98% <= 267 milliseconds
    99.98% <= 268 milliseconds
    99.98% <= 269 milliseconds
    99.98% <= 270 milliseconds
    99.99% <= 271 milliseconds
    99.99% <= 272 milliseconds
    99.99% <= 273 milliseconds
    99.99% <= 274 milliseconds
    99.99% <= 275 milliseconds
    99.99% <= 864 milliseconds
    99.99% <= 865 milliseconds
    99.99% <= 866 milliseconds
    99.99% <= 867 milliseconds
    99.99% <= 868 milliseconds
    100.00% <= 869 milliseconds
    100.00% <= 870 milliseconds
    100.00% <= 871 milliseconds
    100.00% <= 872 milliseconds
    100.00% <= 873 milliseconds
    100.00% <= 873 milliseconds
    110853.69 requests per second
    
    redis-benchmark -h server -p 6380 -t set -n 10000000 -P 256 -c 768
    ====== SET ======
      10000000 requests completed in 4.47 seconds
      768 parallel clients
      3 bytes payload
      keep alive: 1
    
    0.00% <= 2 milliseconds
    0.00% <= 9 milliseconds
    0.02% <= 10 milliseconds
    0.06% <= 11 milliseconds
    0.17% <= 12 milliseconds
    0.29% <= 13 milliseconds
    0.44% <= 14 milliseconds
    1.33% <= 15 milliseconds
    2.41% <= 16 milliseconds
    3.83% <= 17 milliseconds
    5.83% <= 18 milliseconds
    8.42% <= 19 milliseconds
    11.30% <= 20 milliseconds
    14.10% <= 21 milliseconds
    16.94% <= 22 milliseconds
    19.60% <= 23 milliseconds
    22.31% <= 24 milliseconds
    25.01% <= 25 milliseconds
    27.65% <= 26 milliseconds
    30.40% <= 27 milliseconds
    33.02% <= 28 milliseconds
    35.74% <= 29 milliseconds
    38.77% <= 30 milliseconds
    41.45% <= 31 milliseconds
    44.18% <= 32 milliseconds
    46.89% <= 33 milliseconds
    49.36% <= 34 milliseconds
    52.06% <= 35 milliseconds
    54.80% <= 36 milliseconds
    57.48% <= 37 milliseconds
    60.42% <= 38 milliseconds
    63.25% <= 39 milliseconds
    65.84% <= 40 milliseconds
    68.45% <= 41 milliseconds
    71.05% <= 42 milliseconds
    73.70% <= 43 milliseconds
    76.34% <= 44 milliseconds
    78.99% <= 45 milliseconds
    81.75% <= 46 milliseconds
    84.61% <= 47 milliseconds
    87.32% <= 48 milliseconds
    89.78% <= 49 milliseconds
    91.89% <= 50 milliseconds
    93.38% <= 51 milliseconds
    94.47% <= 52 milliseconds
    95.22% <= 53 milliseconds
    95.40% <= 54 milliseconds
    95.55% <= 55 milliseconds
    95.70% <= 56 milliseconds
    95.81% <= 57 milliseconds
    95.87% <= 58 milliseconds
    95.91% <= 59 milliseconds
    95.94% <= 60 milliseconds
    95.99% <= 61 milliseconds
    96.03% <= 62 milliseconds
    96.05% <= 63 milliseconds
    96.08% <= 64 milliseconds
    96.09% <= 253 milliseconds
    96.09% <= 259 milliseconds
    96.12% <= 260 milliseconds
    96.13% <= 261 milliseconds
    96.14% <= 262 milliseconds
    96.17% <= 263 milliseconds
    96.20% <= 264 milliseconds
    96.23% <= 265 milliseconds
    96.29% <= 266 milliseconds
    96.35% <= 267 milliseconds
    96.40% <= 268 milliseconds
    96.47% <= 269 milliseconds
    96.52% <= 270 milliseconds
    96.60% <= 271 milliseconds
    96.68% <= 272 milliseconds
    96.76% <= 273 milliseconds
    96.81% <= 274 milliseconds
    96.87% <= 275 milliseconds
    96.92% <= 276 milliseconds
    96.96% <= 277 milliseconds
    97.02% <= 278 milliseconds
    97.08% <= 279 milliseconds
    97.12% <= 280 milliseconds
    97.16% <= 281 milliseconds
    97.21% <= 282 milliseconds
    97.27% <= 283 milliseconds
    97.32% <= 284 milliseconds
    97.38% <= 285 milliseconds
    97.43% <= 286 milliseconds
    97.49% <= 287 milliseconds
    97.53% <= 288 milliseconds
    97.58% <= 289 milliseconds
    97.64% <= 290 milliseconds
    97.68% <= 291 milliseconds
    97.72% <= 292 milliseconds
    97.75% <= 293 milliseconds
    97.78% <= 294 milliseconds
    97.80% <= 295 milliseconds
    97.82% <= 296 milliseconds
    97.84% <= 297 milliseconds
    97.85% <= 298 milliseconds
    97.87% <= 299 milliseconds
    97.88% <= 300 milliseconds
    97.89% <= 301 milliseconds
    97.92% <= 302 milliseconds
    97.93% <= 303 milliseconds
    97.94% <= 304 milliseconds
    97.95% <= 305 milliseconds
    97.96% <= 306 milliseconds
    97.96% <= 307 milliseconds
    97.97% <= 308 milliseconds
    97.98% <= 309 milliseconds
    97.98% <= 310 milliseconds
    97.99% <= 311 milliseconds
    98.00% <= 312 milliseconds
    98.00% <= 313 milliseconds
    98.00% <= 315 milliseconds
    98.01% <= 316 milliseconds
    98.01% <= 317 milliseconds
    98.02% <= 318 milliseconds
    98.02% <= 320 milliseconds
    98.02% <= 321 milliseconds
    98.03% <= 322 milliseconds
    98.03% <= 323 milliseconds
    98.03% <= 326 milliseconds
    98.04% <= 327 milliseconds
    98.04% <= 328 milliseconds
    98.04% <= 830 milliseconds
    98.04% <= 832 milliseconds
    98.05% <= 834 milliseconds
    98.05% <= 835 milliseconds
    98.06% <= 836 milliseconds
    98.06% <= 837 milliseconds
    98.08% <= 838 milliseconds
    98.10% <= 839 milliseconds
    98.10% <= 840 milliseconds
    98.11% <= 841 milliseconds
    98.12% <= 842 milliseconds
    98.13% <= 843 milliseconds
    98.14% <= 844 milliseconds
    98.15% <= 845 milliseconds
    98.16% <= 846 milliseconds
    98.17% <= 847 milliseconds
    98.19% <= 848 milliseconds
    98.21% <= 849 milliseconds
    98.24% <= 850 milliseconds
    98.26% <= 851 milliseconds
    98.28% <= 852 milliseconds
    98.31% <= 853 milliseconds
    98.33% <= 854 milliseconds
    98.35% <= 855 milliseconds
    98.39% <= 856 milliseconds
    98.44% <= 857 milliseconds
    98.47% <= 858 milliseconds
    98.50% <= 859 milliseconds
    98.53% <= 860 milliseconds
    98.55% <= 861 milliseconds
    98.58% <= 862 milliseconds
    98.61% <= 863 milliseconds
    98.64% <= 864 milliseconds
    98.68% <= 865 milliseconds
    98.71% <= 866 milliseconds
    98.74% <= 867 milliseconds
    98.77% <= 868 milliseconds
    98.80% <= 869 milliseconds
    98.85% <= 870 milliseconds
    98.89% <= 871 milliseconds
    98.92% <= 872 milliseconds
    98.96% <= 873 milliseconds
    98.97% <= 874 milliseconds
    99.00% <= 875 milliseconds
    99.03% <= 876 milliseconds
    99.05% <= 877 milliseconds
    99.07% <= 878 milliseconds
    99.08% <= 879 milliseconds
    99.09% <= 880 milliseconds
    99.12% <= 881 milliseconds
    99.15% <= 882 milliseconds
    99.18% <= 883 milliseconds
    99.22% <= 884 milliseconds
    99.26% <= 885 milliseconds
    99.27% <= 886 milliseconds
    99.29% <= 887 milliseconds
    99.30% <= 888 milliseconds
    99.32% <= 889 milliseconds
    99.33% <= 890 milliseconds
    99.35% <= 891 milliseconds
    99.36% <= 892 milliseconds
    99.38% <= 893 milliseconds
    99.40% <= 894 milliseconds
    99.41% <= 895 milliseconds
    99.42% <= 896 milliseconds
    99.43% <= 897 milliseconds
    99.45% <= 898 milliseconds
    99.48% <= 899 milliseconds
    99.51% <= 900 milliseconds
    99.54% <= 901 milliseconds
    99.58% <= 902 milliseconds
    99.61% <= 903 milliseconds
    99.64% <= 904 milliseconds
    99.67% <= 905 milliseconds
    99.71% <= 906 milliseconds
    99.75% <= 907 milliseconds
    99.78% <= 908 milliseconds
    99.82% <= 909 milliseconds
    99.85% <= 910 milliseconds
    99.86% <= 911 milliseconds
    99.86% <= 913 milliseconds
    99.87% <= 915 milliseconds
    99.87% <= 916 milliseconds
    99.88% <= 918 milliseconds
    99.88% <= 919 milliseconds
    99.89% <= 920 milliseconds
    99.90% <= 921 milliseconds
    99.91% <= 922 milliseconds
    99.91% <= 924 milliseconds
    99.92% <= 927 milliseconds
    99.92% <= 928 milliseconds
    99.92% <= 930 milliseconds
    99.93% <= 931 milliseconds
    99.93% <= 932 milliseconds
    99.93% <= 934 milliseconds
    99.93% <= 935 milliseconds
    99.94% <= 936 milliseconds
    99.95% <= 937 milliseconds
    99.95% <= 938 milliseconds
    99.97% <= 939 milliseconds
    99.97% <= 940 milliseconds
    99.98% <= 941 milliseconds
    99.98% <= 942 milliseconds
    99.99% <= 943 milliseconds
    100.00% <= 943 milliseconds
    2236136.00 requests per second
    
  • TLS support

    TLS support

    @tidwall , I have added TLS support on a fork because I needed it for a project I'm working on, is there any interest for that feature to be merged into this repo?

  • Redis CLI requires `COMMAND` command on connect

    Redis CLI requires `COMMAND` command on connect

    While this is not technically a limitation of this library, but as I was implementing a server with redcon I noticed that my local redis-cli client v5.0.7 didn't seem to work with the redcon-based server I made. After doing some digging I noticed that the redis-cli issues a COMMAND command after a connection is established and it expects a response to work. Once I responded to that with redcon the official redis-cli worked fine. Not sure if you want to add this as a not in the README if people attempt to use it and it doesn't work as expected right away.

  • what's the max concurrent subscriber redcon can take?

    what's the max concurrent subscriber redcon can take?

    Would like to understand how much memory is allocated for each connection. If you can share some insights to what to look out for for high concurrent connections and how to improve etc, will be great. Have yet to check out the internal mechanics of the code.

  • Discovered a bug

    Discovered a bug

    <?php
    
            $qrediswp = new Redis();
            $qrediswp->connect("127.0.0.1", 6379); //dashboard host, port
    
    //hset "e" "432232157184245456" "123456789"
    
            $lss_disksize = $qrediswp->hget("e","432232157184245456");
            echo '= '.$lss_disksize.' =';
    
    // it shows :123456789 instead of 123456789.
    why is there a ":" in front of the value? it doesnt show this using the real redis server.
    
            exit;
    

    Tried with this code...

    //shows up using this:
    
                    case "hget":
    
                            if bytes.Compare(cmd.Args[1],[]byte("e"))==0 { //disk space
                                            conn.WriteUint64(uint64(123456789))
    
                           }
    
    
  • Would pub/sub support be welcome?

    Would pub/sub support be welcome?

    Would I be welcome to implement pub/sub and send a pull request to this repository?

    I am in need of a pub/sub right now and if a pull request is welcome, I will implement and send it.

  • Add server.Addr() : returns server's listen address.

    Add server.Addr() : returns server's listen address.

    Useful to get actual listen address when we listen to random port (localhost:0).

    Listening to random port is very useful in unit test

    Also move the s.ln=ln assignment from serve to ListenServeAndSignal to avoid race condition.

  • memory leaking when connection is reset with go 1.11.x

    memory leaking when connection is reset with go 1.11.x

    I built a service using redcon and deployed it in production environment.

    One day, for some reason (maybe unstable internal networking), massive connection reset occurred, I observed that reset connections are not released and memory keep leaking.

    I recompiled my binary with go 1.10.x and 1.9.x, everything is ok, reset connections are released correctly and no memory leaks.

    Problem only occur when compiled with go 1.11.x.

    I'd investigate this issue and report any progress later.

  • C#: StackExchange.Redis is Unable to connect to REDCON

    C#: StackExchange.Redis is Unable to connect to REDCON

    hi @tidwall

    After compiled the example, Go: redigo can connect to the example server, but C#: StackExchange.Redis can't and C#: StackExchange.Redis can connect to tile38 server.

    In order to connect to redcon with C#: StackExchange.Redis, what should I do?

  • Graceful shutdown

    Graceful shutdown

    Would be nice for the project I'm working on to have a way to gracefully shutdown a redcon server as we run 1 with TLS and one without.

    This is not a priority feature so no idea how long this would take. @tidwall , will open a pull request when done if interested.

  • error handling differ between redcon & redis 3.0.6/3.2.4

    error handling differ between redcon & redis 3.0.6/3.2.4

    The following simple script shows a difference in behavior between redcon and redis 3.0.6/3.2.4:

    `#!/bin/bash

    TIMESTAMP=$(TZ=UTC date +"%Y-%m-%dT%H:%M:%S.%3NZ") HOSTNAME=$(hostname -s)

    ROLE=myrole APPLICATION=myapp MESSAGE="Ping" REDISKEY="testkey" REDIS_SVR="localhost 7379"

    PAYLOAD='{ "@fields": { "application": "'${APPLICATION}'", "role": "'${ROLE}'" }, "@message": "'${MESSAGE}'", "@source_host": "'${HOSTNAME}'", "@timestamp": "'${TIMESTAMP}'" }'

    ( echo "RPUSH ${REDISKEY} '"${PAYLOAD}"'" sleep 1s ) | telnet $REDIS_SVR`

    redcon, in a sense correctly answers:

    -ERR Protocol error: unbalanced quotes in request

    redis 3.0.6/3.2.4 however has no gripes with the situation:

    :1

  • Race condition using redcon v1.6.0

    Race condition using redcon v1.6.0

    Hello, there is a race condition in the redcon library used by olric v0.5.0. I was testing my code using go test -race and this warning appeared.

    Sample

    	olricInstance = config.New("local")
    	db, err := olric.New(olricInstance)
    	if err != nil {
    		return nil, err
    	}
    
    	go func() {
    		_ = db.Start()
    	}()
    

    Output

    WARNING: DATA RACE
    Read at 0x00c0000360a8 by goroutine 19:
      github.com/tidwall/redcon.(*Writer).Flush()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:651 +0x3e
      github.com/tidwall/redcon.(*conn).Close()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:459 +0x47
      github.com/tidwall/redcon.serve.func1.1()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:338 +0x119
      github.com/tidwall/redcon.serve.func1()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:341 +0x56
      runtime.deferreturn()
          /usr/local/go/src/runtime/panic.go:476 +0x32
      github.com/tidwall/redcon.(*Server).Serve()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:311 +0x155
      github.com/buraksezer/olric/internal/server.(*Server).ListenAndServe()
          /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/internal/server/server.go:189 +0x784
      github.com/buraksezer/olric.(*Olric).Start.func1()
          /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:324 +0x47
      golang.org/x/sync/errgroup.(*Group).Go.func1()
          /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x86
    
    Previous write at 0x00c0000360a8 by goroutine 37:
      github.com/tidwall/redcon.(*Writer).Flush()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:654 +0xbe
      github.com/tidwall/redcon.handle.func2()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:438 +0x237
      github.com/tidwall/redcon.handle()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:442 +0xb8
      github.com/tidwall/redcon.serve.func2()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:378 +0x47
    
    Goroutine 19 (running) created at:
      golang.org/x/sync/errgroup.(*Group).Go()
          /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x12e
      github.com/buraksezer/olric.(*Olric).Start()
          /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:323 +0x2a4
      command-line-arguments.EmbeddedOlricConnectionFactory.func3()
          /Users/sylvain/go/src/github.com/darkweak/souin/cache/providers/embeddedOlricProvider.go:86 +0x39
    
    Goroutine 37 (running) created at:
      github.com/tidwall/redcon.serve()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:378 +0x89b
      github.com/tidwall/redcon.(*Server).Serve()
          /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:311 +0x155
      github.com/buraksezer/olric/internal/server.(*Server).ListenAndServe()
          /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/internal/server/server.go:189 +0x784
      github.com/buraksezer/olric.(*Olric).Start.func1()
          /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:324 +0x47
      golang.org/x/sync/errgroup.(*Group).Go.func1()
          /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x86
    
  • race when close redcon

    race when close redcon

    redcon version: 1.4.4

    POC

    package main
    
    import (
    	"context"
    	"log"
    	"strings"
    
    	"github.com/go-redis/redis/v9"
    	"github.com/tidwall/redcon"
    )
    
    func main() {
    	redcon := redcon.NewServerNetwork("tcp", "127.0.0.1:12345",
    		func(conn redcon.Conn, cmd redcon.Command) {
    			switch strings.ToLower(string(cmd.Args[0])) {
    			case "set":
    				log.Println(string(cmd.Args[1]), string(cmd.Args[2]))
    				conn.WriteString("OK")
    			default:
    				log.Println("cmd not implemented: ", string(cmd.Args[0]))
    				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
    			}
    
    		}, nil, nil,
    	)
    	ch := make(chan error)
    	go func() {
    		redcon.ListenServeAndSignal(ch)
    	}()
    	if err := <-ch; err != nil {
    		log.Fatal(err)
    	}
    
    	// create redis client, test set command
    	client := redis.NewClient(&redis.Options{
    		Addr:    redcon.Addr().String(),
    		Network: redcon.Addr().Network(),
    	})
    	r := client.Set(context.Background(), "tmp", "test", 0)
    	if r.Err() != nil {
    		log.Fatal(r.Err())
    	}
    	if err := client.Close(); err != nil {
    		log.Fatal(err)
    	}
    
    	// close redcon (race)
    	if err := redcon.Close(); err != nil {
    		log.Fatal(err)
    	}
    }
    
    // go run -race poc.go
    
    

    The race occurs when closing redcon while the handle function is not finished (invoking c.wr.Flush, but without lock).

    image
  • Type safety

    Type safety

    Sometimes we need to guarantee the arguments are of some type (for example GET requires name to be string but what if user put in number?) so how do we check it?

  • use a reusable slice for storing args in readcommands functions

    use a reusable slice for storing args in readcommands functions

    Unsure if this is usable to you or users of your library since it only works on newer Go versions due to the use of Generics but I figured creating a PR for this wouldn't hurt.

  • Request for comment on this line

    Request for comment on this line

    Thank you so much for your work on this tool.

    Could you please add a comment to this line? https://github.com/tidwall/redcon/blob/52d396ed1ef10ebff46177d5d2fbc059b3df8227/redcon.go#L28

    Is it to ensure buffer cap is bounded? Curious to understand if you have seen an issue in production that triggered this change.

    Thanks!

An HTTP client for go-server-timing middleware. Enables automatic timing propagation through HTTP calls between servers.

client-timing An HTTP client for go-server-timing middleware. Features: An HTTP Client or RoundTripper, fully compatible with Go's standard library. A

Dec 24, 2022
A simple and lightweight encrypted password manager written in Go.
A simple and lightweight encrypted password manager written in Go.

A simple and lightweight encrypted password manager written in Go.

Jun 16, 2022
Remark42 is a self-hosted, lightweight, and simple comment engine
Remark42 is a self-hosted, lightweight, and simple comment engine

Remark42 is a self-hosted, lightweight, and simple (yet functional) comment engine, which doesn't spy on users. It can be embedded into blogs, articles or any other place where readers add comments.

Dec 28, 2022
Dead simple rate limit middleware for Go.

Limiter Dead simple rate limit middleware for Go. Simple API "Store" approach for backend Redis support (but not tied too) Middlewares: HTTP, FastHTTP

Jan 7, 2023
Simple middleware to rate-limit HTTP requests.

Tollbooth This is a generic middleware to rate-limit HTTP requests. NOTE 1: This library is considered finished. NOTE 2: Major version changes are bac

Jan 4, 2023
A dead simple parser package for Go
A dead simple parser package for Go

A dead simple parser package for Go

Jan 8, 2023
Go middleware for monetizing your API on a per-request basis with Bitcoin and Lightning ⚡️

ln-paywall Go middleware for monetizing your API on a per-request basis with Bitcoin and Lightning ⚡️ Middlewares for: net/http HandlerFunc net/http H

Jan 6, 2023
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

Its task is simple: It tells you which language some provided textual data is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages.

Dec 29, 2022
Stash is a locally hosted web-based app written in Go which organizes and serves your porn.

Stash is a locally hosted web-based app written in Go which organizes and serves your porn.

Jan 2, 2023
Simple rule based matchmaking for your online game with support of Redcon(RESP) protocol.
 Simple rule based matchmaking for your online game with support of Redcon(RESP) protocol.

Simple Matchmaking Simple rule based matchmaking for your online game with support of Redcon(RESP) protocol. 1- Simple Match Rule Easiest usage of sys

Jan 4, 2023
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.

GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right

Apr 27, 2022
Golog is a logger which support tracing and other custom behaviors out of the box. Blazing fast and simple to use.

GOLOG Golog is an opinionated Go logger with simple APIs and configurable behavior. Why another logger? Golog is designed to address mainly two issues

Oct 2, 2022
JPRQ Customizer is a customizer that helps to use the JPRQ server code and make it compatible with your own server with custom subdomain and domain
JPRQ Customizer is a customizer that helps to use the JPRQ server code and make it compatible with your own server with custom subdomain and domain

JPRQ Customizer is a customizer that helps to use the JPRQ server code and make it compatible with your own server with custom subdomain and domain.You can upload the generated directory to your web server and expose user localhost to public internet. You can use this to make your local machine a command center for your ethical hacking purpose ;)

Jan 19, 2022
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具,满足用户非常灵活的同步、迁移需求。
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具,满足用户非常灵活的同步、迁移需求。

RedisShake is mainly used to synchronize data from one redis to another. Thanks to the Douyu's WSD team for the support. 中文文档 English tutorial 中文使用文档

Dec 29, 2022
Use Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.
Use  Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.

目录 gRPC/consul/kafka简介 gRPC+kafka的Demo gRPC+kafka整体示意图 限流器 基于redis计数器生成唯一ID kafka生产消费 kafka生产消费示意图 本文kafka生产消费过程 基于pprof的性能分析Demo 使用pprof统计CPU/HEAP数据的

Jul 9, 2022
Rpcx-framework - An RPC microservices framework based on rpcx, simple and easy to use, ultra fast and efficient, powerful, service discovery, service governance, service layering, version control, routing label registration.

RPCX Framework An RPC microservices framework based on rpcx. Features: simple and easy to use, ultra fast and efficient, powerful, service discovery,

Jan 5, 2022
A fast, easy-of-use and dependency free custom mapping from .csv data into Golang structs

csvparser This package provides a fast and easy-of-use custom mapping from .csv data into Golang structs. Index Pre-requisites Installation Examples C

Nov 14, 2022
Use pingser to create client and server based on ICMP Protocol to send and receive custom message content.
Use pingser to create client and server based on ICMP Protocol to send and receive custom message content.

pingser Use pingser to create client and server based on ICMP Protocol to send and receive custom message content. examples source code: ./examples Us

Nov 9, 2022
Golang-redis-webserver - Web server using redis

Web Server using Redis Api REST Database SQLITE3 Cache Redis # Creating record s

Jun 19, 2022
A TCP Server Framework with graceful shutdown, custom protocol.

xtcp A TCP Server Framework with graceful shutdown,custom protocol. Usage Define your protocol format: Before create server and client, you need defin

Dec 7, 2022