A memcached binary protocol toolkit for go.

gomemcached

This is a memcached binary protocol toolkit in go.

It provides client and server functionality as well as a little sample server showing how I might make a server if I valued purity over performance.

Server Design

overview

The basic design can be seen in gocache. A storage server is run as a goroutine that receives a MCRequest on a channel, and then issues an MCResponse to a channel contained within the request.

Each connection is a separate goroutine, of course, and is responsible for all IO for that connection until the connection drops or the dataServer decides it's stupid and sends a fatal response back over the channel.

There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.

Comments
  • added support for SASL auth commands

    added support for SASL auth commands

    this is required to allow go-couchbase to write to buckets other than default

    all the changes from "dustin" to "mschoch" are unfortunate, but i didn't see a good way around that to easily test my changes and eventually send a pull request. perhaps you can suggest a better workflow.

    also, this is my first go.

  • Client howto?

    Client howto?

    Hi,

    First thanks for this piece of code (and for memcache). You seems to be the only go memcache binary client out there!

    I'm just wondering if i should use only one connection (memcached.Connect) and use it in every goroutine (are connections goroutine safe?) or should i use one connection per goroutine or ...

    Also is the readme still accurate (There is currently no work at all in making the thing perform (there are specific areas I know need work). This is just my attempt to learn the language somewhat.)?

    Are you or other people succesfully using it "in production"?

    Thanks in advance

  • Return the most up-to-date CAS on success.

    Return the most up-to-date CAS on success.

    Normally CAS isn't checked in the success cases, except when waiting on data to be persisted. Peviously the library failed to update the CAS value on success causing WriteUpdate calls with the Index/Persist flags set to always fail with ErrOverwritten.

    These were false errors, as the library was always using the pre-modification CAS value as input into the observe function instead of the post-modification CAS value.

  • Updating a key resets it's expiry

    Updating a key resets it's expiry

    https://github.com/dustin/gomemcached/blob/master/client/mc.go#L389

    Why is this done? It appears to ignore the expiry value passed in and reset it to 0.

    In our system, this leads to updated keys never expiring. The fix is easy, but wondering why this was done in the first place?

  • Installation of GoMemcached client fails

    Installation of GoMemcached client fails

    When installing go-couchbase, it installs this package as well and it fails saying this.

    # github.com/dustin/gomemcached/client
    /usr/lib/go/src/pkg/github.com/dustin/gomemcached/client/mc.go:339: function ends without a return statement
    /usr/lib/go/src/pkg/github.com/dustin/gomemcached/client/tap_feed.go:110: function ends without a return statement
    
  • Implement basic opcode dispatch

    Implement basic opcode dispatch

    Right now, all commands return "unknown command". It should be trivial to implement a new module that dispatches based on opcode lookup in a map or even array.

  • License

    License

    Hi! What about license for the code? Please, add one. I think BSD-style license is a good choise. Go (http://golang.org/LICENSE) and a lot of go libraries use it.

  • Missing null check on transmit

    Missing null check on transmit

    Saw this crash:

    github.com/dustin/gomemcached/client.(*Client).Transmit(0x0, 0xf8401c52d0, 0xf8401bf740, 0x1e0000001e, 0xf8402bb610, ...)
            /Users/dustin/go/src/github.com/dustin/gomemcached/client/mc.go:68 +0x29
    github.com/dustin/gomemcached/client.(*Client).GetBulk(0x0, 0x3d9, 0xf8402c9000, 0x100000001, 0x0, ...)
            /Users/dustin/go/src/github.com/dustin/gomemcached/client/mc.go:231 +0x349
    github.com/couchbaselabs/go-couchbase.(*Bucket).doBulkGet(0xf8400fd500, 0xf8401003d9, 0xf8402c9000, 0x100000001, 0xf84014e5a0, ...)
            /Users/dustin/go/src/github.com/couchbaselabs/go-couchbase/client.go:179 +0x171
    created by github.com/couchbaselabs/go-couchbase.(*Bucket).GetBulk
            /Users/dustin/go/src/github.com/couchbaselabs/go-couchbase/client.go:210 +0x2fb
    
  • fix read body len bug when key and extras len is zero

    fix read body len bug when key and extras len is zero

    Hi, Dustin: First of all, please forgive me for not being good in English. Memcached binary protocol: Increment, Decrement, version have response:

    • MUST NOT have extras.
    • MUST NOT have key.
    • MUST have value.

    This bug causes the body bytes not to be read. version doc

  • Avoid allocating new slice for data on every Receive()

    Avoid allocating new slice for data on every Receive()

    In MCRequest.Receive, a new buffer is allocated for every receive for the body of the request:

    buf := make([]byte, klen+elen+bodyLen)
    

    It would be great if this function could instead accept a buffer as an argument so that buffers could be re-used across requests. The code could use cap(buf) to ensure that the buffer is long enough, and buf = buf[:klen+elen+bodyLen] to limit its length. Only if it buf is too small should it allocate its own (and potentially warn the caller that the buffer was not used).

  • [client] Unknown command

    [client] Unknown command

    This is the error message:

    MCResponse status=UNKNOWN_COMMAND, opcode=SASL_LIST_MECHS,
     opaque=0, msg: Unknown command
    

    This is the code:

    
    func Memcache() {
        client, err := memcached.Connect("tcp", "127.0.0.1:11211")
        if err != nil {
            log.Fatalf("Error connecting: %v", err)
        }
    
        resp, err := client.Auth("memcache", "password")
        if err != nil {
            log.Fatalf("auth error: %v", err)
        }
        log.Printf("Auth response = %v", resp)
    }
    
Related tags
A memcached proxy that manages data chunking and L1 / L2 caches
A memcached proxy that manages data chunking and L1 / L2 caches

Rend: Memcached-Compatible Server and Proxy Rend is a proxy whose primary use case is to sit on the same server as both a memcached process and an SSD

Dec 24, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Dec 29, 2022
Go Memcached client library #golang

About This is a memcache client library for the Go programming language (http://golang.org/). Installing Using go get $ go get github.com/bradfitz/gom

Dec 28, 2022
memcached operator

memcached-operator Operator SDK 中的 Go 编程语言支持可以利用 Operator SDK 中的 Go 编程语言支持,为 Memcached 构 建基于 Go 的 Operator 示例、分布式键值存储并管理其生命周期。 前置条件 安装 Docker Desktop,

Sep 18, 2022
🧩 Redify is the optimized key-value proxy for quick access and cache of any other database throught Redis and/or HTTP protocol.

Redify (Any database as redis) License Apache 2.0 Redify is the optimized key-value proxy for quick access and cache of any other database throught Re

Sep 25, 2022
Tpf2-tpnetmap-toolkit - A toolkit to create svg map images from TransportFever2 world data
Tpf2-tpnetmap-toolkit - A toolkit to create svg map images from TransportFever2 world data

tpf2-tpnetmap-toolkit TransportFever2 のワールドデータから svg のマップ画像を作成するツールキットです。 1. 導入方

Feb 17, 2022
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache Summary groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many case

Dec 31, 2022
A memcached proxy that manages data chunking and L1 / L2 caches
A memcached proxy that manages data chunking and L1 / L2 caches

Rend: Memcached-Compatible Server and Proxy Rend is a proxy whose primary use case is to sit on the same server as both a memcached process and an SSD

Dec 24, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Dec 29, 2022
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

groupcache Summary groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many case

Dec 29, 2022
Go Memcached client library #golang

About This is a memcache client library for the Go programming language (http://golang.org/). Installing Using go get $ go get github.com/bradfitz/gom

Jan 8, 2023
Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)

gokv Simple key-value store abstraction and implementations for Go Contents Features Simple interface Implementations Value types Marshal formats Road

Dec 24, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Dec 29, 2022
Go Memcached client library #golang

About This is a memcache client library for the Go programming language (http://golang.org/). Installing Using go get $ go get github.com/bradfitz/gom

Dec 28, 2022
memcached operator

memcached-operator Operator SDK 中的 Go 编程语言支持可以利用 Operator SDK 中的 Go 编程语言支持,为 Memcached 构 建基于 Go 的 Operator 示例、分布式键值存储并管理其生命周期。 前置条件 安装 Docker Desktop,

Sep 18, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Jan 3, 2023
searchHIBP is a golang tool that implements binary search over a hash ordered binary file.

searchHIBP is a golang tool that implements binary search over a hash ordered binary file.

Nov 9, 2021
Package pbpgx provides a toolkit for easier Protocol Buffers interaction with PostgreSQL databases.

PBPGX Package pbpgx provides a toolkit for easier Protocol Buffers interaction with PostgreSQL databases. Pbpgx supports the Protocol Buffer types gen

Jun 27, 2022
Golang binary for data exfiltration with ICMP protocol
Golang binary for data exfiltration with ICMP protocol

QueenSono ICMP Data Exfiltration A Golang Package for Data Exfiltration with ICMP protocol. QueenSono tool only relies on the fact that ICMP protocol

Jan 3, 2023
An experiment building a custom binary protocol for a calculator

Overview Implementation of a calculator service built on a custom protocol on top of TCP Details The server is in main.go, and the client is in client

Nov 28, 2021