go websocket, a better way to buid your IM server

Your star is my power!! 🚀

License MIT Go Report Card GoDoc Awesome

Discribe

lhttp is a http like protocol using websocket to provide long live, build your IM service quickly scalable without XMPP!

Everything is customizable.

简体中文

Features

  • simple easy but powerful!
  • fast, publish 10000 messages using 0.04s(single-core CPU,1G memory).
  • support cluster.
  • easy to customize and expansion.
  • work well with HTTP. So LHTTP can work with others language like PHP java python etc,.

A simple chat room demo

chat-demo with lhttp javascript sdk we complete a simple chat room within 40 lines code!!

SDKs

Header filter development

Protocol stack:

+--------------------+
|       lhttp        |
+--------------------+
|     websocket      |
+--------------------+
|        TCP         |
+--------------------+

Architecture

        +---------------------------------------+
        |    message center cluster (gnatsd)    |
        +---------------------------------------+
 ........|.................|...............|..................
| +-------------+   +-------------+   +-------------+        | 
| |lhttp server |   |lhttp server |   |lhttp server |   ...  |  lhttp server cluster
| +-------------+   +-------------+   +-------------+        | 
 .....|..........._____|  |___.............|  |_________......
      |          |            |            |            |       <----using websocket link
 +--------+  +--------+   +--------+   +--------+   +--------+   
 | client |  | client |   | client |   | client |   | client |   
 +--------+  +--------+   +--------+   +--------+   +--------+  

Quick start

go get github.com/nats-io/nats
go get github.com/fanux/lhttp

We need run gnatsd first:

cd bin
./gnatsd &
./lhttpd 

Open anohter bash run lhttpClient, then input your command:

cd bin
./lhttpClient

Ship on docker

$ docker build -t lhttp:latest .
$ docker run -p 9090:9090 -p 8081:8081 lhttp:latest

Open two windows in your browser, enter http://localhost:9090.

Lhttp server port is 8081, your own websocket client can connect to ws://localhost:8081

Enjoy the chat...

Alternative, pull image from docker hub.

$ docker run -p 9090:9090 -p 8081:8081 fanux/lhttp:latest

Protocol

LHTTP/1.0 Command\r\n                --------start line, define command, and protocol [protocol/version] [command]\r\n
Header1:value\r\n                    --------headers
Header2:value\r\n
\r\n
body                                 --------message body

for example:

LHTTP/1.0 chat\r\n
content-type:json\r\n
publish:channel_jack\r\n
\r\n
{
    to:jack,
    from:mike,
    message:hello jack,
    time:1990-1210 5:30:48
}

Usage

define your processor, you need combine BaseProcessor

type ChatProcessor struct {
    *lhttp.BaseProcessor
}

if you don't like BaseProcessor, define your struct witch must has OnOpen(*WsHandler) OnClose(*WsHandler) method like this:(don't recommand)

type ChatProcessor struct {
}
func (p ChatProcessor)OnOpen(h *WsHandler) {
    //your logic
}
func (p ChatProcessor)OnClose(h *WsHandler) {
    //your logic
}
func (p ChatProcessor)OnMessage(h *WsHandler) {
    //your logic
}

regist your processor

lhttp.Regist("chat",&ChatProcessor{&lhttp.BaseProcessor{}})

then if command is "chat" ChatProcessor will handle it

define your onmessage handle

func (p *ChatProcessor)OnMessage(h *WsHandler) {
    h.Send(h.GetBody())
}

Start websocket server

http.Handler("/echo",lhttp.Handler(lhttp.StartServer))
http.ListenAndServe(":8081")

Example , echo

type ChatProcessor struct {
    *lhttp.BaseProcessor
}

func (p *ChatProcessor) OnMessage (h *lhttp.WsHandler) {
    log.Print("on message :", h.GetBody())
    h.Send(h.GetBody())
}

func main(){
    lhttp.Regist("chat", &ChatProcessor{&lhttp.BaseProcessor{}})

    http.Handle("/echo",lhttp.Handler(lhttp.StartServer))
    http.ListenAndServe(":8081",nil)
}

Test

open websocketServer and run:

cd websocketServer
go run test.go

as we can see, both of the new headers are added and new command is set by the server. If we don't set a header or command ,then they will return the same result as they requested.

open an other bash, and run client in websocketClient

cd websocketClient
go run test.go

Subscribe/Publish

client1:

LHTTP/1.0 command\r\n
subscribe:channelID\r\n
\r\n
body optional

client2:

LHTTP/1.0 command\r\n
publish:channelID\r\n
\r\n
body require

client1:

LHTTP/1.0 command\r\n
unsubscribe:channelID\r\n
\r\n
body optional

client2 publish a message by channelID, client1 subscribe it, so client 1 will receive the message. if client1 send unsubscribe channelID, he will not receive message any more in channelID

support multiple channelID:

LHTTP/1.0 chat\r\n
subscribe:channelID1 channelID2 channelID3\r\n
\r\n

Using HTTP publish message!

lhttp support publish message by standard HTTP. URL: /publish . method: POST . body: use lhttp publishes message as HTTP body. for example I want send a message to who subscribe channel_test by HTTP.

    resp,err := http.POST("https://www.yourserver.com/publish", "text/plain",
    "LHTTP/1.0 chat\r\npublish:channel_test\r\n\r\nhello channel_test guys!")

when lhttp server receive this message, will publish whole body to channel_test.

your can use Publish function in tools.go

//func Publish(channelID []string, command string, header map[string]string, body string) (err error) {
//}
//send message to who subscribe mike.

Publish("mike", "yourCommand", nil, "hello mike!")

Upstream

we can use lhttp as a proxy:

LHTTP/1.0 command\r\n
upstream:POST http://www.xxx.com\r\n
\r\n
body

lhttp will use hole message as http body, post to http://www.xxx.com if method is GET, lhttp send http GET request ignore lhttp message body:

LHTTP/1.0 command\r\n
upstream:GET http://www.xxx.com?user=user_a&age=26\r\n
\r\n
body

This case will show you about upstream proxy:

jack use lhttp chat with mike, lhttp is third part module, we can't modify lhttp server but we want to save the chat record, how can we do?

        +----+                  +----+
        |jack|                  |mike|
        +----+                  +----+
         |_____________    _______|
                       |  |
                   +------------+
                   |lhttp server|
                   +------------+
                         |(http request with chat record)
                         V
                   +------------+
                   | http server|  upstream server(http://www.xxx.com/record)
                   +------------+
                   (save chat record)
    

jack: MESSAGE_UPSTREAM

LHTTP/1.0 chat\r\n
upstream:POST http://www.xxx.com/record\r\n
publish:channel_mike\r\n
\r\n
hello mike,I am jack

mike:

LHTTP/1.0 chat\r\n
subscribe:channel_mike\r\n
\r\n

when jack send publish message, not only mike will receive the message, the http server will also receive it. witch http body is:MESSAGE_UPSTREAM, so http server can do anything about message include save the record

Multipart data

for example a file upload message, the multipart header record the offset of each data part, each part can have it own headers

LHTTP/1.0 upload\r\n
multipart:0 56\r\n
\r\n
content-type:text/json\r\n
\r\n
{filename:file.txt,fileLen:5}
content-type:text/plain\r\n
\r\n
hello
content-type:text/json\r\n\r\n{filename:file.txt,fileLen:5}content-type:text/plain\r\n\r\nhello
^                                                          ^
|<---------------------first part------------------------->|<---------second part------------>|
0                                                          56                           

why not boundary but use offset? if use boundary lhttp need ergodic hole message, that behaviour is poor efficiency. instead we use offset to cut message

How to get multipart data

for example this is client message.

LHTTP/1.0 upload\r\nmultipart:0 14\r\n\r\nk1:v1\r\n\r\nbody1k2:v2\r\n\r\nbody2

server code:

type UploadProcessor struct {
	*lhttp.BaseProcessor
}

func (*UploadProcessor) OnMessage(ws *lhttp.WsHandler) {
	for m := ws.GetMultipart(); m != nil; m = m.GetNext() {
		log.Print("multibody:", m.GetBody(), " headers:", m.GetHeaders())
	}
}

//don't forget to regist your command processor

lhttp.Regist("upload", &UploadProcessor{&lhttp.BaseProcessor{}})

Partners

Owner
中弈
A crazy cloud compute coder! 阿里内部想用sealos的欢迎找我 @中弈
中弈
Comments
  • 使用http协议发布消息问题

    使用http协议发布消息问题

    使用http协议发布消息问题,使用和文档中的方式类进行调用,调用失败 http.POST("http://127.0.0.1:8581/publish", "text/plain", "LHTTP/1.0 chat\r\npublish:channel_test\r\n\r\nhello channel_test guys!") 直接调用不通。 换成 http.POST("http://127.0.0.1:8581/chat", "text/plain", "LHTTP/1.0 chat\r\npublish:channel_test\r\n\r\nhello channel_test guys!") 返回:bad method

  • run error!

    run error!

    hello,Could you describe this command in detail?

    “We need run gnatsd first: cd bin ./gnatsd & ./lhttpd ......”

    gnatsd & lhttpd is not a command

  • wsHandler.go里面的buildMessage没有做防错

    wsHandler.go里面的buildMessage没有做防错

    如果消息是: "LHTTP/1.0 chat\r\n \r\n nihao" 那么必然会在range里面报错 数组下标越界 这边应该做个处理

    //parse websocket body func buildMessage(data string) *WsMessage { //TODO optimise ,to use builder pattern s := data message := &WsMessage{message: data} message.headers = make(map[string]string, headerMax) //parse message

    //parse start line
    i := strings.Index(s, CRLF)
    message.command = s[protocolLength+1 : i]
    
    //parse hearders
    k := 0
    headers := s[i+2:]
    var key string
    var value string
    //traverse once
    for j, ch := range headers {
    	if ch == ':' && key == "" {
    		key = headers[k:j]
    		k = j + 1
    	} else if headers[j:j+2](这边) == CRLF {
    		value = headers[k:j]
    		k = j + 2
    
    		message.headers[key] = value
    		// log.Print("parse head key:", key, " value:", value)
    		key = ""
    	}
    	if headers[k:k+2] (这边)== CRLF {
    		k += 2
    		break
    	}
    }
    
    //set body
    message.body = headers[k:]
    
    return message
    

    }

  • Please add a LICENSE file to this project, the demo and the JS-SDK

    Please add a LICENSE file to this project, the demo and the JS-SDK

    This looks like an awesome project but one thing that is missing so that other people can decide whether to use it, is a license. Thanks for your time & have a nice day.

  • 多个lhttp server的问题

    多个lhttp server的问题

    为了支撑大量用户(例如20万用户)的websocket连接,我部署多个lhttp server,例如有lhttp server1和lhttp server2,如果client1通过websocket连接到lhttp server1,client2通过websocket连接到lhttp server2,client2 subscribe了client1的message,那client1 publish到lhttp server1的message如何让client2接收到?谢谢。

  • 关于unnsubscribe不了的问题

    关于unnsubscribe不了的问题

    我使用websocketServer里面的test .go做服务器,然后启动websocketClient里面的test.go做客户端进行测试,我nsubscribe了camera_123后并且publish一个消息后camera_123可以接收到,但是我unsubscribe了camera_123后还是可以接收到消息,这样好像不对吧。谢谢。

  • add maxlength limit

    add maxlength limit

    • // RegistHeadFilter(&upstreamHeadFilter{})
    • // RegistHeadFilter(&multipartFilter{})

    增加单个包大小限制,减轻服务器转发压力 (目前为40K,在 lhttpDefine.go 中定义) 恢复被错误注释掉的包头处理....

  • Fix broken headings in Markdown files

    Fix broken headings in Markdown files

    GitHub changed the way Markdown headings are parsed, so this change fixes it.

    See bryant1410/readmesfix for more information.

    Tackles bryant1410/readmesfix#1

  • 如何关闭这个socket

    如何关闭这个socket

    如何关闭socket

        initLHTTP() {
          this.lHttpClient = new Lhttp("ws://127.0.0.1:9527");
          console.log(this.lHttpClient);
        },
        lHTTPOnOpen() {
          if (this.visible === true) {
            this.lHttpClient.on_open = function(context) {
              context.subscribe("test", "", null, "");
            };
          } else {
            this.lHttpClient.conn.close();
          }
        },
        lHTTPOnMessage() {
          this.lHttpClient.on_message = function(context) {
            if (context.getBody() !== "") {
              const lhttpData = JSON.parse(context.getBody().split("\r\n")[0]);
              console.log(lhttpData);
            }
          };
        },
        lHTTPOnError() {
          this.lHttpClient.on_error = function() {
            this.initLHTTP();
          };
        },
        lHTTPOnClose() {
          this.lHttpClient.on_close = function(context) {
            console.log("onclose:" + this.lHttpClient.conn + context);
            this.lHttpClient.conn.close();
          };
        },
    

    这样写lHTTPOnClose不生效,我只能把close写到

        lHTTPOnOpen() {
          if (this.visible === true) {
            this.lHttpClient.on_open = function(context) {
              context.subscribe("test", "", null, "");
            };
          } else {
            this.lHttpClient.conn.close();
          }
        },
    
  • 关于单机lhttp服务器的websocket最大连接数

    关于单机lhttp服务器的websocket最大连接数

    单机lhttp服务器的websocket最大连接数我按照https://www.jianshu.com/p/e0b52dc702d6的步骤优化系统的参数进行测试只有51322个,使用的服务器是dell 730xd,32g内存,测试软件使用https://github.com/changhu2013/websocket_bench。我想做到单机支撑10万个以上的websocket连接数,请问哪里可以再优化?谢谢。

This package helps establish a websocket connection to the bilibili streaming server.

biliStreamClient This package helps establish a websocket connection to the bilibili streaming server. bilibili直播弹幕的WebSocket协议分析请参考:https://blog.csdn

Oct 25, 2022
Starting my way through learning Golang by setting up an HTTP server.

Lets-Go Setting up an HTTP server with Golang. Building a simple server with "net/http" library in Golang. This is a simpe server with two routes, the

Aug 22, 2022
It is a proxy to improve article readability, a directory for your favorite articles, and a way to make the internet lighter and more accessible.

timoneiro It is a work in progress. Some features are unimplemented yet. The helmsman's goal is to be a way to browse articles without all the distrac

Jun 13, 2022
Go-komoot - An easy way to communicate your user with Komoot

Go Komoot library This is an easy way to communicate your user with Komoot. Via

Feb 5, 2022
开箱即用的基于命令的消息处理框架,让 websocket 和 tcp 开发就像 http 那样简单

Cmd Srv 开箱即用的基于命令的消息处理框架,让 websocket 和 tcp 开发就像 http 那样简单

Sep 25, 2022
Minimal and idiomatic WebSocket library for Go

websocket websocket is a minimal and idiomatic WebSocket library for Go. Install go get nhooyr.io/websocket Highlights Minimal and idiomatic API First

Dec 30, 2022
A modern, fast and scalable websocket framework with elegant API written in Go
A modern, fast and scalable websocket framework with elegant API written in Go

About neffos Neffos is a cross-platform real-time framework with expressive, elegant API written in Go. Neffos takes the pain out of development by ea

Jan 4, 2023
HTTP, HTTP2, HTTPS, Websocket debugging proxy
HTTP, HTTP2, HTTPS, Websocket debugging proxy

English | 简体中文 We recommend updating whistle and Node to ensure that you receive important features, bugfixes and performance improvements. Some versi

Dec 31, 2022
WebSocket Connection Smuggler
WebSocket Connection Smuggler

ws-smuggler ws-smuggler is websocket connection smuggling testing tool. It is similar to the this project, but it has been rewritten based on the web

Jan 3, 2023
websocket proxy,简单的websocket反向代理实现,支持ws、wss
websocket proxy,简单的websocket反向代理实现,支持ws、wss

websocket proxy 100行代码实现轻量的websocket代理库,不依赖其他三方库,支持ws、wss代理 使用示例 Install go get github.com/pretty66/websocketproxy import ( "github.com/pretty66/w

Dec 27, 2022
High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

Jan 8, 2023
Websocket proxy component
Websocket proxy component

Proxy server component Task description Task description is in DESCRIPTION.md Issues found in task description and fixed signal.Notify wasn't cathing

Dec 1, 2022
NotifyTool - A message forwarding service for http to websocket

notifyTool this is a message forwarding service for http to websocket task webso

Jan 3, 2022
Pubsub-go - Go-redis pubsub with websocket

go-redis pubsub with websocket # start a local dev server $ make dev

Jan 28, 2022
Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo.

go-chat Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo. Why Eu gostaria de aprender algumas ferramenta

Jul 14, 2022
PlanB: a HTTP and websocket proxy backed by Redis and inspired by Hipache.

PlanB: a distributed HTTP and websocket proxy What Is It? PlanB is a HTTP and websocket proxy backed by Redis and inspired by Hipache. It aims to be f

Mar 20, 2022
HTTP tunnel over Websocket
HTTP tunnel over Websocket

WS PROXY This is a reverse HTTP proxy over websockets. The aim is to securely make call to internal APIs from outside. How does it works a WSP client

Nov 12, 2022