Adding power support.

Adding power support ppc64le This is part of the Ubuntu distribution for ppc64le. This helps us simplify testing later when distributions are re-building and re-releasing. For more info tag @gerrith3.

Add variadic options to zk.Connect, and a HostProvider interface based on the Java code. An initial DNSHostprovider is the default, matching Java's implementation.

Background:

The channel approach causes lost events during period of scheduling contention. Of particular harm is the loss of HasSession since it's usually the state that triggers actions.

The callback approch puts the responsiblity on the caller of the library to decide on the tradeoff between losing events and blocking the main event loop.

I found a race condition which meant that you could call receive on a nil channel and thus cause a thread to wait indefinitely. Proposed fix is to pass an already made channel to addWatcher() rather than return one.

In case when reconnection occurs, it is necessary to re-authenticate with ZK because it "forgets" the authentication data. Otherwise, the connection is unusable with messages like below popping up in the logs:

2016/08/29 12:08:36 [ERR] core: failed to write seal configuration: zk: not authenticated
2016/08/29 12:08:46 [ERR] core: failed to write seal configuration: zk: not authenticated
2016/08/29 12:08:56 [ERR] core: failed to write seal configuration: zk: not authenticated

The problem can be reproduced with this simple program:

package main

import (
    "fmt"
    "time"

    "github.com/samuel/go-zookeeper/zk"
)

func main() {
    c, _, err := zk.Connect([]string{"127.0.0.1"}, time.Second) //*10)
    if err != nil {
        panic(err)
    }
    defer c.Close()

    for c.State() != zk.StateHasSession {
        fmt.Println("State: ", c.State())
        time.Sleep(time.Second)
    }
    fmt.Printf("Connected to %s\n", c.Server())

    c.AddAuth("digest", []byte("userfoo:passbar"))

    acl := zk.DigestACL(zk.PermAll, "userfoo", "passbar")

    _, err = c.Create("/dir", nil, 0, acl)
    if err != nil && err != zk.ErrNodeExists {
        panic(err)
    }

    for {
        res_path, err := c.Create("/dir/dupa", []byte("maryna"), zk.FlagEphemeral|zk.FlagSequence, acl)
        if err != nil {
            fmt.Printf("Error creating ephemeral node: %s\n", err)
        } else {
            fmt.Printf("Created node %s\n", res_path)
        }
        time.Sleep(time.Second)
    }
}

One only needs to block tcp port 2181 traffic or pause/unpause ZK container/process.

I have not found official ZK protocol specification but instead based my work on what ZK Python client Kazoo does: https://github.com/python-zk/kazoo/blob/master/kazoo/client.py#L722 The change basically stores all submitted auth credentials in connection structure, and on reconnect the sendLoop is stalled until all authentication data has been resubmitted. The re-submission happens synchronously, without involving sendLoop. After all data has been submitted, send loop starts and submits all queued requests. The recieveLoop is unchanged due to the fact that outstanding responses are purged on reconnect.

This patch may probably be still improved (not re-submitting the same auth data, de-duplicating auth data for digest schema, etc...) and I am not sure if this is the right approach, but I think this is a step into right direction :)

I would be grateful for feedback.

Thanks! pr

Java and other clients will validate on the client side before sending it on. Although this should be handled server side, there are apparently some implemenations that don't do this server side, so this follows suit with the other client libraries.

Fixes #63

I have just started using Multi call and have run into two issues.

Issues:

1. When a Multi call is preformed with zookeeper, it seems I do not get a response. -No sure what is causing this I will dig a little deeper and see what comes up.
2. Because I never get a response, I never return from the call. -Their should be a time out on the receive channel, do you want me to add it?

Supersedes #205

• changes travis tests to handle all versions of the ZK binary distribution.
• Adding new APIs IncrementalReconfig and Reconfig to support 3.5 reconfig options.

tested on ZK 3.5.4-beta 5 node zk clusters with simple add and remove actions beyond the integration tests included in this PR.

Fixes #204

This fixes an issue wherein a client that has a very large number of watches (and/or very long path names that are being watched) could cause a disconnect to be effectively permanent: the client gets stuck in a loop of trying to reconnect but being continually disconnected by the server.

This happens because the client may try to construct a request that is too large for the server to process. A ZK server, by default, limits incoming requests to 1mb. The hard-coded request buffer size in this client is 1.5mb.

It also fixes a related possible bug where the connection is successfully re-established, but watches are silently lost (because the set-watches request is so big it results in ErrShortBuffer when trying to encode the set-watches-request packet).

Introduces a connection option that allows limiting the size of response packets that are buffered in the client. This matches the implementation of Jute in Java, which is the IDL and serialization format ZK uses.

Sometimes zookeeper just drops connection on invalid session data, we prefer to drop session and start from scratch when that event occurs instead of dropping into loop of connect/disconnect attempts.

This can be emulated by stopping zookeeper, cleaning it's database and starting it again. But we experience same bug in production without cleaning zookeeper, but in case of network partition (not every partition event).

Reproduce this case when test my app with go race. The log:

2021/06/21 15:21:58 recv loop terminated: err=failed to read from connection: read tcp 192.168.1.101:56952->192.168.1.101:2181: i/o timeout
2021/06/21 15:21:58 recv loop terminated: err=failed to read from connection: read tcp 192.168.1.101:56976->192.168.1.101:2181: i/o timeout
==================
WARNING: DATA RACE
Write at 0x00c0001ee000 by goroutine 77:
  github.com/samuel/go-zookeeper/zk.(*Conn).recvLoop()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:910 +0xc4c
  github.com/samuel/go-zookeeper/zk.(*Conn).loop.func2()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:521 +0x2f1

Previous read at 0x00c0001ee000 by goroutine 8:
  github.com/samuel/go-zookeeper/zk.(*Conn).sendSetWatches()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:637 +0x472
  github.com/samuel/go-zookeeper/zk.(*Conn).loop()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:535 +0x5a4
  github.com/samuel/go-zookeeper/zk.Connect.func1()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:223 +0x3c

Goroutine 77 (running) created at:
  github.com/samuel/go-zookeeper/zk.(*Conn).loop()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:516 +0x576
  github.com/samuel/go-zookeeper/zk.Connect.func1()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:223 +0x3c

Goroutine 8 (running) created at:
  github.com/samuel/go-zookeeper/zk.Connect()
      /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:222 +0x8c6
      
     // ... more application call stack
     // call like this
     // zkConn, chanConnect, err := zk.Connect([]string{zkAddr}, time.Second*30, zk.WithLogInfo(false))
==================

centos 7 go 1.16.2 zookeeper 3.6.2

Originally posted by @huaz836 in https://github.com/samuel/go-zookeeper/issues/178#issuecomment-864974329

I found this line in conn.go

https://github.com/samuel/go-zookeeper/blob/master/zk/conn.go#L717 c.conn.SetReadDeadline(time.Time{})

In my view, if there occurs a problem in the network or server, clear the timeout deadline may cause this goroutine stuck for a long time even forever, then others stuck too because of the channel.

I wonder why it was written like this, was there some reason I've missed it? If necessary, I'd like to post an mr.

as source code below, if data of path change , it will trigger three event. why not only trigger watchTypeData

func (c *Conn) recvLoop(conn net.Conn) error { .... case EventNodeDeleted, EventNodeDataChanged: wTypes = append(wTypes, watchTypeExist, watchTypeData, watchTypeChild)

.... }

The current method can only create nodes one level at a time, adding the method of creating nodes step by step. If there is no parent node, create the parent node first