I am currently looking into implementing a TCP-in-TCP connection tunneling protocol very similar to qmux or OpenSSH in Devolutions Gateway + jetsocat, a project written in Rust. The cleaned up protocol specification used in qmux is exactly what I'm looking for, but I've noticed that the message headers haven't been modified to respect memory alignment rules commonly enforced in certain types of processors like ARM. To read a uint32 (4-bytes) then it should be at a memory address that is a multiple of 4 bytes, etc. This is the primary reason why a lot of protocols try hard to align their data structures, and why compilers automatically pad data structures that don't respect those rules. You can still read data structures that are not memory aligned, but it comes with a performance cost.

I know this would be a breaking change, but would you be interested in making the change anyway? I will likely implement the memory aligned variant on my side and keep everything else as-is. Here is what the current header structure looks like:

  byte      QMUX_MSG_CHANNEL_OPEN
  uint32    sender channel
  uint32    initial window size
  uint32    maximum packet size

And what a memory-aligned variant would look like:

  byte      QMUX_MSG_CHANNEL_OPEN
  byte     reserved[3]
  uint32    sender channel
  uint32    initial window size
  uint32    maximum packet size

This would free 1 byte and a uint16 for other use like flags, etc, if you have such needs. What do you think? Everything else that follows would be nicely memory-aligned with those 3 bytes.

This is the meat of the implementation. Although I'd like better test coverage and comments, I'm particularly concerned about the use of contexts. I don't believe they are used where given or appropriate where given so I'd like to think about them more.

Also, I don't often centralize errors into predefined variables, but it might be a good idea to here?

Double check min-max limits ... there are packet sizes and data packet sizes and I'm not sure they're correctly used?

After this there is simply a transport subpackage with various transport protocol dialers and such.

If synchronous review works better, schedule a time with me!

Debugged some issues with reads blocking.

This still has a bunch of debugging logs and hacky code, but putting it up for some feedback on my attempts to test this.

These are just from a single client connection: Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50518: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50524: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50528: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50550: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50552: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50558: i/o timeout Failed to extract vhost name: read tcp 45.76.255.14:4415->173.44.95.183:50562: i/o timeout

Client and Server running on two different linux systems

Server DNS has wildcard subdomains set Server bound to 0.0.0.0 Server port 4415

Running client on two different PCs simultaneously Client connects to server with port 3415 Running quick test using mini-httpd listening on port 3415 with simple 2-line output html file

Browsers using generated address to connect to other PC successfully - html displayed

Plan is to use groktunnel to do crypto wallet transfers (EPIC Cash) to wallet on LAN behind router (using ngrok currently but don't trust them to keep the free access going so we need our own option).

Load issue with server throwing constant errors even if sending to /dev/nul ?

Can Server handle thousands of concurrent client connections?

I tried a wallet transfer from a mining pool to local wallet (these transactions were fine with localtunnel). It's a very small data stream. The server threw a Failed to extract vhost name: EOF and ended the session. Client shows EOF and Exit. So it looks like groktunnel is very http specific and will not allow just the http header then an open tcp tunnel like localtunnel. Will have to use that instead.

I switched my internet connection recently and they use Carrier Grade NAT.

1. If I install this on a Linode server, can I install the same code on my local server (Raspberry PI) to enable incoming connections by talking to my Linode server running the same?
2. Also does this mean, all traffic from outside my home via this grok server (Linode) to my home PI will go through Linode? Or is it just network translation? I am concerned about possible bandwidth usage.

Adds a type parameter to queue to help with type checking, and updates the method signatures to include undefined where appropriate.

Also adds specific types to Array<Function>.

Looks like there were some API changes that weren't integrated into the codec test.

This looks like it was meant to test what's now Unmarshal and Marshal. Since Unmarshal returns a Message which only has ID, that doesn't match AnyMessage which corresponds to more specific types. The cast here doesn't seem ideal, but should I change the return type or parameter to be compatible?

Adds tests for the current transports.

I'm getting a spurious failure for the Pipe-based IO test that I still need to investigate if it's specific to that test implementation, or something else:

--- FAIL: TestIO (0.00s)
    --- FAIL: TestIO/channel_read (0.00s)
        qmux/golang/transport/transport_test.go:43: io: read/write on closed pipe

Start adding a couple additional tests for session package:

• context timeout on Open
• Session.Wait

Also marks fatal as a test helper so that errors show the caller's line number instead of the line inside fatal.

Rather nice and readable. Could use some tests for error cases, but otherwise >90% coverage.

How important are godoc comments with the spec right there?

Implements a TypeScript transport based on Deno websockets.

Extracts the buffering of data for pending reads into a ReadBuffer class to reuse.

Fixes #17

Hi! Thanks for this project! it looks awesome.

I'm trying to run my own instance on VPS using docker. i can establish connection between client and server. i get back a randomized url but when in try to use it i get this kind of error:

go_srv_1  | 2022/01/26 17:25:10 groktunnel server [grok.greenponik.com] ready!
go_srv_1  | 2022/01/26 17:25:11 75npf5e8b1.grok.greenponik.com:9999: start session
go_srv_1  | 2022/01/26 17:25:11 8m0bcmc5cm.grok.greenponik.com:9999: start session
go_srv_1  | Failed to extract vhost name: invalid method "\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03\xf3\xf2V\x9f;dx\xb9y]\t&\x10Q\xcbL\x97\x0f\xd9"

On server side: i just change default value for port, host and addr in main.go (args aren't takes when i use them on Dockerfile script) main.go:

        var port = flag.String("p", "9999", "server port to use")
        var host = flag.String("h", "grok.greenponik.com", "server hostname to use")
        var addr = flag.String("b", "54.37.11.133", "ip to bind [server only]")
        flag.Parse()

here my Dockerfile:

FROM golang:1.16-alpine

WORKDIR /app

COPY ./go.mod .
COPY ./go.sum .
RUN go mod download

COPY . /app

RUN go build

CMD ["./groktunnel"]

here my docker-compose.yml:

version: "3.8"

services:
  go_srv:
    build:
      context: .
      dockerfile: Dockerfile.server
    image: go_srv
    network_mode: host
    restart: always

on client side: main.go:

	var port = flag.String("p", "9999", "server port to use")
	var host = flag.String("h", "grok.greenponik.com", "server hostname to use")
	var addr = flag.String("b", "127.0.0.1", "ip to bind [server only]")
	flag.Parse()

Dockerfile:

WORKDIR /app

COPY ./go.mod .
COPY ./go.sum .
RUN go mod download

COPY . /app

RUN go build

CMD ["./groktunnel", "8000"]

docker-compose.yml:

version: "3.8"

services:
  tunnel:
    build:
      context: .
      dockerfile: Dockerfile
    image: go_srv
    restart: always

How an i debug this error? is it about the vhost "github.com/inconshreveable/go-vhost" component?

this way transport/index.ts can check for deno and import/re-export the deno module transports, otherwise import/re-export the browser module transports.