Golang push server cluster

gopush-cluster

gopush-cluster is a go push server cluster.

Features

  • light weight
  • high performance
  • pure golang implementation
  • message expired
  • offline message store
  • public message or private message push
  • multiple subscribers (can restrict max subscribers)
  • heartbeat(service heartbeat or tcp keepalive)
  • auth (if a subscriber not auth then can't connect to comet node)
  • multiple protocol (websocket, tcp, todo http longpolling)
  • stat
  • cluster support (easy add or remove comet & web & message node)
  • failover support (zookeeper)

Architecture

gopush-cluster

Document

English

中文

LICENSE

gopush-cluster is is distributed under the terms of the GNU General Public License, version 3.0 GPLv3

Comments
  • golang中多goroutine操作map时直接赋值不用加锁?

    golang中多goroutine操作map时直接赋值不用加锁?

    你好, 看到rand_lb.go中有这样的代码: r.Clients在这里未加锁直接修改r.Clients = tmpClients,在其他地方会被读取,这种赋值需要加锁吧?难道是高级用法?

    tmpClients := make(map[string]*WeightRpc, len(r.Clients))
            for addr, client := range r.Clients {
                if client == nil {
                    continue
                }
                tmpClients[addr] = client
                if client.Addr == retryAddr {
                    client.Client = rpcTmp
                }
            }
            // atomic update clients
            r.Clients = tmpClients
    
  • no title

    no title

    Just wondering how can I connect to the push-cluster using web socket or tcp since I can not find any sample code here.

    Say, I am looking to implement a chatroom that users can pub/sub messages via web socket or tcp. How can I do this with the help of gopush-cluster? If I implement it on my own, the code should be like this:

    netConn, err := net.Dial("tcp", "127.0.0.1:9999")
    ws, _, err := websocket.NewClient(netConn, URL, nil, readBuffer, writeBuffer)
    # read message
    err = ws.ReadJSON(&v)
    # write message
    err = ws.WriteJSON(&v)
    

    I mean, what should I do in the client side to connect to the gopush-cluster and receive, send messages?

  • 关于Migrate的疑惑

    关于Migrate的疑惑

    首先很感谢这个库,由于这个库我知晓了不少分布式架构知识。 另外这个库是我学习golang以来研究的第一份源码,有个地方有些疑惑,希望能得到回答。

    关于Migrate, 我查看cometchannel.go看到应该是comet在接到Migrate消息后nodeWeightMap更新到最新,然后去检查当前记录的channel key的一致性hash是否是本节点,如果不是就移除并且关闭此channel对应的所有长连接。 上述理解,应该没问题吧。

    我非常疑惑的是通知Migrate的时机,是由comet节点Add事件以外的事件触发的(Del和Update)。rpc/comet.go L195

    一个一致性hash环,不是应该被删除的节点上的key会被分散到其他节点。而添加节点的话,会从其他节点里匀出一部分到新节点么,那为什么在新节点出现的时候不去通知已有节点把该匀出来那些key清理掉?

    如果是节点被删除,那其他正常节点也没必要有清理操作啊,他们本身已有的那些key并不会因为某节点的删除而产生所属变化啊。

    可能我的问题比较小白,还请见谅,在下对golang和分布式都是初学,想了很久也没搞懂这个问题。

  • 第7步配置貌似 文件不起作用

    第7步配置貌似 文件不起作用

    $ nohup ./message -c message.conf & $ nohup ./comet -c comet.conf & $ nohup ./web -c web.conf &

    我查看到8090断网未监听,而 80端口监听了。 还有,只监听localhost 换成非127 ip无法连接。修改配置文件,换成0.0.0.0 也不是监听所所有。

  • 按照步骤无法正常跑通测试

    按照步骤无法正常跑通测试

    hi, 我是在一台centos 6.4 x86_64环境下进行搭建的。

    按照步骤 可以跑起来message web comet三个进程

    [root@vagrant-centos64 bin]# netstat -apn | grep web tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN 16761/./web tcp 0 0 127.0.0.1:8091 0.0.0.0:* LISTEN 16761/./web tcp 0 0 127.0.0.1:8190 0.0.0.0:* LISTEN 16761/./web tcp 0 0 127.0.0.1:48539 127.0.0.1:6970 ESTABLISHED 16761/./web tcp 0 0 127.0.0.1:50988 127.0.0.1:2181 ESTABLISHED 16761/./web tcp 0 0 127.0.0.1:36149 127.0.0.1:8070 ESTABLISHED 16761/./web [root@vagrant-centos64 bin]# netstat -apn | grep comet tcp 0 0 127.0.0.1:6968 0.0.0.0:* LISTEN 16754/./comet tcp 0 0 127.0.0.1:6969 0.0.0.0:* LISTEN 16754/./comet tcp 0 0 127.0.0.1:6970 0.0.0.0:* LISTEN 16754/./comet tcp 0 0 127.0.0.1:6971 0.0.0.0:* LISTEN 16754/./comet tcp 0 0 127.0.0.1:6972 0.0.0.0:* LISTEN 16754/./comet tcp 0 0 127.0.0.1:6970 127.0.0.1:48539 ESTABLISHED 16754/./comet tcp 0 0 127.0.0.1:36147 127.0.0.1:8070 ESTABLISHED 16754/./comet tcp 0 0 127.0.0.1:50986 127.0.0.1:2181 ESTABLISHED 16754/./comet [root@vagrant-centos64 bin]# netstat -apn | grep message tcp 0 0 127.0.0.1:8170 0.0.0.0:* LISTEN 16748/./message tcp 0 0 127.0.0.1:8070 0.0.0.0:* LISTEN 16748/./message tcp 0 0 127.0.0.1:8070 127.0.0.1:36149 ESTABLISHED 16748/./message tcp 0 0 127.0.0.1:8070 127.0.0.1:36147 ESTABLISHED 16748/./message tcp 0 0 127.0.0.1:50985 127.0.0.1:2181 ESTABLISHED 16748/./message

    测试结果:

    [root@vagrant-centos64 bin]# curl -d "test2" 'http://localhost:8091/admin/push/public?expire=600' 404 page not found

    其他相关服务: root 4612 1 0 11:05 pts/1 00:00:10 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,CONSOLE -cp /data/programfiles/zookeeper-3.4.5/bin/../build/classe root 4671 1 0 11:07 pts/0 00:00:03 /etc/init.d/redis-server /etc/redis/redis.conf root 4708 1819 0 11:16 pts/0 00:00:00 sudo su - root 4709 4708 0 11:16 pts/0 00:00:00 su - root 4710 4709 0 11:16 pts/0 00:00:00 -bash root 16480 1 0 13:15 pts/0 00:00:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld mysql 16582 16480 0 13:15 pts/0 00:00:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/r root 16748 4710 0 13:44 pts/0 00:00:00 ./message -c message.conf nobody 16754 4710 0 13:44 pts/0 00:00:00 ./comet -c comet.conf root 16761 4710 0 13:45 pts/0 00:00:00 ./web -c web.conf

    求指教 谢谢

  • go语言的版本是否有些过老,在新版本下编译不过(go 1.7.3)

    go语言的版本是否有些过老,在新版本下编译不过(go 1.7.3)

    $ go version go version go1.7.3 windows/386

    $ go install github.com/terry-mao/gopush-cluster/web

    github.com/Terry-Mao/gopush-cluster/zk

    ..\zk\zk.go:148: undefined: syscall.Kill

    希望可以适配golang娇高的版本

  • Redis.Dial()报错,没有解析到正确的Redis服务器

    Redis.Dial()报错,没有解析到正确的Redis服务器

    Message模块配置Redis为不带密码的情况下,会跳转到Messge/redis.go的91行 log.Error("redis.Dial("%s", "%s") error(%v)", tmpProto, tmpAddr, err) 变量tmpProto和tmpAddr都是空

  • gopush-cluster-javascript-sdk在index.html页websocket 403错误

    gopush-cluster-javascript-sdk在index.html页websocket 403错误

    在学习lz的这套框架。搭建起来运行正常。准备用js sdk测试一下websocket。遇到一个问题: WebSocket connection to 'ws://localhost:6968/sub?key=Lanfei&heartbeat=60' failed: Error during WebSocket handshake: Unexpected response code: 403

    应该怎么解决?

  • 有选择的存储消息,避免占用过多内存

    有选择的存储消息,避免占用过多内存

    如果所有消息都存储到redis,占用的内存可能比较大。

    我们的场景:

    1. gopush的消息存储,不适合做永久消息历史之用,我们自己会做消息历史的存储。
    2. 发送个人消息的时候,如果commet发现有订阅,就不用告知message去存储了
    3. 离线消息一旦被请求,也不必再次存储。这样每次获取离线消息,也无需传递消息id。

    我不知道,上述场景是否通用,能否实现?比如通过配置文件设置,或者增加请求参数来实现?

    谢谢

  • 65535 错误

    65535 错误

    当执行这句话的时候。即使没有Terry-Mao 这个用户,报404 比较好。 curl -d "test" 'http://localhost:8091/admin/push?key=Terry-Mao&expire=600&gid=0'

    {"ret":65535}

    E0502 09:12:52.124653 20465 redis.go:87] json.Marshal("&{[116 101 115 116] 1398993772}") error(json: error calling MarshalJSON for type json.RawMessage: invalid character 'e' in literal true (expecting 'r')) E0502 09:12:52.125464 20465 rpc.go:60] UseStorage.SavePrivate("Terry-Mao", "test", 13989931721237542, 600) error(json: error calling MarshalJSON for type json.RawMessage: invalid character 'e' in literal true (expecting 'r')) E0502 09:12:52.125878 20474 seq_channel.go:112] MessageRPC.SavePrivate("Terry-Mao", "&{Terry-Mao [116 101 115 116] 13989931721237542 600}", &ret) error(json: error calling MarshalJSON for type json.RawMessage: invalid character 'e' in literal true (expecting 'r')) E0502 09:12:52.143297 20474 rpc.go:113] ch.PushMsg("Terry-Mao", "&{[116 101 115 116] 13989931721237542 0}") error(json: error calling MarshalJSON for type json.RawMessage: invalid character 'e' in literal true (expecting 'r')) E0502 09:12:52.143899 20312 admin.go:80] client.Call("CometRPC.PushPrivate", "&{Terry-Mao [116 101 115 116] 600}", &ret) error(json: error calling MarshalJSON for type json.RawMessage: invalid character 'e' in literal true (expecting 'r'))

  • Simplify gopush-cluster

    Simplify gopush-cluster

    How can I config gopush-cluster to be used like this :

    Client <-- Comet <-- Web

    I do not want to store any offline data on the server but just want to push data to all connected clients for my app. :-) Maybe it's a bit weird but it's exactly what I want :)

  • user channel list 求余的问题

    user channel list 求余的问题

    //func (l *ChannelList) Bucket(key string) *ChannelBucket 
    idx := uint(h.Sum32()) & uint(Conf.ChannelBucket-1)  //其中 ChannelBucket = runtime.NumCPU()
    

    只有当x=2^n(n为自然数)时,a % x = a & (x - 1 )才成立, 又服务器cpu至少我见过有10核12核的

    那么这里就计算出的idx会有问题吧

  • bug for regexp

    bug for regexp

    file path: message/redis.go

    reg := regexp.MustCompile("(.+)@(.+)#(.+)|(.+)@(.+)")
    

    This match for ""@""# or ""@"", but for the file like *.conf, comment(#) is not required.

    So, it's a good idea for change to the following

    reg := regexp.MustCompile("(.+)@(.+)")
    
  • Modify absolute dependency path to relative and Add mid config

    Modify absolute dependency path to relative and Add mid config

    1. Modify absolute dependency path to relatvie path , when to fork gopush-cluster project, can save project to anywhere, will not dependency origin project .
    2. Add config field mid_use_long_timestamp (default is yes), let user can select long or short timestamp to use. Because redis ZADD score dose not support whole int64, so when use long timestamp as mid, it will lead mid has been changed when save private message to redis, if use short timestamp , it will be work. For some business the short timestamp as mid is enough.
Golang push server cluster
Golang push server cluster

gopush-cluster gopush-cluster is a go push server cluster. Features light weight high performance pure golang implementation message expired offline m

Dec 28, 2022
A push notification server written in Go (Golang).
A push notification server written in Go (Golang).

gorush A push notification micro server using Gin framework written in Go (Golang) and see the demo app. Contents gorush Contents Support Platform Fea

Jan 8, 2023
Uniqush is a free and open source software system which provides a unified push service for server side notification to apps on mobile devices.

Homepage Download Blog/News @uniqush Introduction Uniqush (\ˈyü-nə-ku̇sh\ "uni" pronounced as in "unified", and "qush" pronounced as in "cushion") is

Jan 9, 2023
Simple push notification system (android/fcm, apns/slideshow) written by Golang

Golang Push Notification Simple system push notification for android/fcm and apn

Dec 20, 2021
Bark is an iOS App which allows you to push customed notifications to your iPhone.
Bark is an iOS App which allows you to push customed notifications to your iPhone.

Bark is an iOS App which allows you to push customed notifications to your iPhone.

Jan 3, 2023
May 11, 2023
Go client to reliable queues based on Redis Cluster Streams

Ami Go client to reliable queues based on Redis Cluster Streams. Consume/produce performance Performance is dependent from: Redis Cluster nodes count;

Dec 12, 2022
Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9

Cluster extensions for Sarama, the Go client library for Apache Kafka 0.9 (and later).

Dec 28, 2022
golang long polling library. Makes web pub-sub easy via HTTP long-poll server :smiley: :coffee: :computer:
golang long polling library.  Makes web pub-sub easy via HTTP long-poll server :smiley: :coffee: :computer:

golongpoll Golang long polling library. Makes web pub-sub easy via an HTTP long-poll server. New in v1.1 Deprecated CreateManager and CreateCustomMana

Jan 6, 2023
websocket based messaging server written in golang

Guble Messaging Server Guble is a simple user-facing messaging and data replication server written in Go. Overview Guble is in an early state (release

Oct 19, 2022
Scalable real-time messaging server in language-agnostic way
Scalable real-time messaging server in language-agnostic way

Centrifugo is a scalable real-time messaging server in language-agnostic way. Centrifugo works in conjunction with application backend written in any

Jan 2, 2023
Server-sent live updates: protocol and reference implementation
Server-sent live updates: protocol and reference implementation

Protocol and Reference Implementation Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast

Jan 1, 2023
⚡️ A lightweight service that will build and store your go projects binaries, Integrated with Github, Gitlab, Bitbucket and Bitbucket Server.
⚡️ A lightweight service that will build and store your go projects binaries, Integrated with Github, Gitlab, Bitbucket and  Bitbucket Server.

Rabbit A lightweight service that will build and store your go projects binaries. Rabbit is a lightweight service that will build and store your go pr

Nov 19, 2022
High-Performance server for NATS, the cloud native messaging system.
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

Jan 2, 2023
NanoMDM is a minimalist Apple MDM server heavily inspired by MicroMDM

NanoMDM NanoMDM is a minimalist Apple MDM server heavily inspired by MicroMDM. Getting started & Documentation Quickstart A quick guide to get NanoMDM

Dec 28, 2022
Notifie Server is a learning list notifier

Notifie Server Notifie Server is a learning list notifier. E.g; If you want to l

Dec 24, 2021
Converter EPG XMLTV to OTT-play (by Alex) server side JSON

EPG converter for OTT-play FOSS Описание Инструмент создания телепрограммы для OTT-Play FOSS, использует 1 поток, и буферное чтение из файла, что позв

Jan 6, 2023
A brief demo of real-time plotting with Plotly, Go, and server-sent events
A brief demo of real-time plotting with Plotly, Go, and server-sent events

Golang SSE Demo A brief demo of real-time plotting with Plotly, Go, and server-side events. Overview I first learned about Server-Sent Events from @mr

Nov 28, 2022
graylog-golang is a full implementation for sending messages in GELF (Graylog Extended Log Format) from Go (Golang) to Graylog

graylog-golang is a full implementation for sending messages in GELF (Graylog Extended Log Format) from Go (Golang) to Graylog

Dec 5, 2022