Package rsync contains a native Go rsync implementation.

gokrazy rsync

tests

Package rsync contains a native Go rsync implementation.

Beware: very fresh. Might eat your data. You have been warned!

The only component currently is gokr-rsyncd, a read-only rsync daemon sender-only Go implementation of rsyncd. rsync daemon is a custom (un-standardized) network protocol, running on port 873 by default.

This project accepts contributions as time permits to merge them (best effort).

Existing rsync implementation survey

Language URL Note Max Protocol Server mode?
C WayneD/rsync original “tridge” implementation; I found older versions easier to study 31 yes
C kristapsdz/openrsync OpenBSD, good docs 27 yes
Go gokrazy/rsync → you are here ← 27 yes 🎉
Go jbreiding/rsync-go rsync algorithm no
Go kaiakz/rsync-os only client/receiver 27 no
Go knight42 proxy no
Go c4milo/gsync no
Java APNIC-net/repositoryd archived yes
Java JohannesBuchner/Jarsync archived, internet draft RFC “The rsync Network Protocol” yes
Java perlundq/yajsync yes
C++ gilbertchen/acrosync-library commercial no
Rust sourcefrog/rsyn client, “rsyn is rsync with no c” 27 no

Getting started

To serve the current directory via rsync on localhost:8730, use:

go install github.com/gokrazy/rsync/cmd/gokr-rsyncd
gokr-rsyncd -modulemap=pwd=$PWD

You can then copy the contents of the current directory with clients such as rsync(1):

% rsync -v --archive --port 8730 rsync://localhost/pwd/ quine
receiving file list ... done
created directory quine
./
.git/
[…]
.github/workflows/main.yml
LICENSE
Makefile
README.md
cmd/gokr-rsyncd/rsyncd.go
doc.go
go.mod
go.sum
internal/rsyncd/connection.go
internal/rsyncd/rsyncd.go
interop_test.go

sent 1,234 bytes  received 5,678 bytes  13,824.00 bytes/sec
total size is 666  speedup is 0.10

…or openrsync(1), shown doing a differential update:

% openrsync -v --archive --port 8730 rsync://localhost/pwd/ quine
socket.c:109: warning: connect refused: ::1, localhost
Transfer starting: 369 files
.git/index (1.1 KB, 100.0% downloaded)
Transfer complete: 5.5 KB sent, 1.2 KB read, 666 B file size

Limitations

Bandwidth

In my tests, gokr-rsyncd can easily transfer data at > 6 Gbit/s. The current bottleneck is the MD4 algorithm itself (not sure whether in the “tridge” rsync client, or in gokr-rsyncd). Implementing support for more recent protocol versions would help here, as these include hash algorithm negotiation with more recent choices.

Protocol related limitations

  • xattrs (including acls) was introduced in rsync protocol 30, so is currently not supported.

Supported environments and privilege dropping

Supported environments:

  1. systemd (Linux)
  2. Docker (Linux)
  3. privileged Linux
  4. privileged non-Linux

In all environments, the default instructions will take care that:

  • (On Linux only) The host file system is made read-only for gokr-rsyncd, to guard against accidental data exfiltration.
  • gokr-rsyncd is running without privileges, as user nobody, to limit the scope of what an attacker can do when exploiting a vulnerability.

Known gaps:

  • gokr-rsyncd does not guard against denial of service attacks, i.e. consuming too many resources (connections, bandwidth, CPU, …).

systemd (unprivileged)

We provide a gokr-rsyncd.socket and gokr-rsyncd.service file for systemd. These files enables most of systemd’s security features. You can check by running systemd-analyze security gokr-rsyncd.service, which should result in an exposure level of “0.2 SAFE” as of systemd 249 (September 2021).

First, configure your server flags by creating a systemd service override file:

systemctl edit gokr-rsyncd.service

In the opened editor, change the file to:

[Service]
ExecStart=
ExecStart=/usr/bin/gokr-rsyncd -modulemap=pwd=/etc/tmpfiles.d

Close the editor and install the service using:

systemctl enable --now gokr-rsyncd.socket

Additional hardening recommendations:

Docker (unprivileged)

We provide a Dockerfile for gokr-rsyncd.

docker run \
  --read-only \
  -p 127.0.0.1:8730:8730 \
  -v /etc/tmpfiles.d:/srv/rsync:ro \
  stapelberg/gokrazy-rsync:latest \
    -modulemap=pwd=/srv/rsync

Additional hardening recommendations:

privileged Linux (including gokrazy.org)

When started as root on Linux, gokr-rsyncd will create a mount namespace, mount all configured rsync modules read-only into the namespace, then change into the namespace using chroot(2) and drop privileges using setuid(2).

Tip: you can verify which file system objects the daemon process can see by using ls -l /proc/$(pidof gokr-rsyncd)/root/.

Additional hardening recommendations:

privileged non-Linux (e.g. Mac)

When started as root on non-Linux (e.g. Mac), gokr-rsyncd will drop privileges using setuid(2).

unprivileged with write permission (e.g. from a shell)

To prevent accidental misconfiguration, gokr-rsyncd refuses to start when it detects that it has write permission in any configured rsync module.

Comments
  • Enable server use as a library

    Enable server use as a library

    Meta

    Issue: https://github.com/gokrazy/rsync/issues/12

    Rebase of older PR: https://github.com/gokrazy/rsync/pull/3 (opened new PR because I wanted to use a branch in my fork instead of main, and I don't think you can change branch in an existing PR)

    Example user application

    This works:

    package main
    
    import (
    	"context"
    	"log"
    	"net"
    	"os/signal"
    	"syscall"
    
    	"github.com/gokrazy/rsync/rsyncd"
    )
    
    func main() {
    	ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
    	defer cancel()
    
    	if err := logic(ctx); err != nil {
    		log.Fatal(err)
    	}
    }
    
    func logic(ctx context.Context) error {
    	listener, err := net.Listen("tcp", "0.0.0.0:666")
    	if err != nil {
    		return err
    	}
    
    	rsyncServer, err := rsyncd.NewServer(rsyncd.Module{
    		Name: "default",
    		Path: "/tmp/rsyncd",
    	})
    	if err != nil {
    		return err
    	}
    
    	if err := rsyncServer.Serve(ctx, listener); err != nil {
    		return err
    	}
    
    	log.Println("gracefully exiting")
    
    	return nil
    }
    
    

    Notes on this PR

    • The PR might be easiest to be reviewed by just reading one commit at a time
    • I changed a few log messages to be more user friendly, but those commits can be easily removed if you don't agree with them
    • Tests pass, at least on my machine :laughing:
    • I did a blanket internal -> pkg, which might not be a good idea (I wanted an easy change to get myself started). But the "user code" needs at least config + rsync packages. I can add a commit that makes not-necessary-to-be-public packages back to internal.
    • The most questionable change I did was change the module map concept to a list. It brought quite a few changes. But I think if the rsyncd internally wants a map, it could take a simple list of modules and internally transform it into a map.
      • Despite this bringing many changes, I think end result is simpler
      • Configuration doesn't have to compute the map from list of modules
    • rsyncd package forcefully logs to stderr. I would like to have Logger *log.Logger field in rsyncd.Server which NewServer() maybe sets to log.Default() by default (preserving the current behaviour) or has to be given explicitly (have not thought of this yet), so user can give discarding logger to make the library quiet. But that could be a new PR, there is quite a bunch of stuff here already.

    General notes

    With my fresh eyes on this codebase, a few notes:

    • "maincmd" as a concept was confusing to me. I knew there is "client" and "server". I was thinking "which one main refers to?".
    • There is name "receiver" which currently means the "client" portion. But that name might not be long-lived if both the client and the server both gain send/receive capabilities. Would name change to client/server remove these ambiquities? Something like:
      • maincmd = servermaincmd
      • receivermaincmd = clientmaincmd
      • rsyncd could also be renamed to rsyncserver, but that's probably not important, the "d" already communicates the server portion..
  • logger usage in `rsyncd` package

    logger usage in `rsyncd` package

    Hey, thanks for creating this great package! I am trying to import the rsyncd package in one of my project. I am trying to create the rsyncd server directly. However, seems like the rsyncd server is using the builtin logger package:

    https://github.com/gokrazy/rsync/blob/de1cff573bd22217f6d5669b89a70f84c3a51829/rsyncd/rsyncd.go#L9

    And it's calling fatal during the execution:

    https://github.com/gokrazy/rsync/blob/c271006f2d3fa756367a856165d349862ae44aa6/rsyncd/filoio.go#L104-L105

    Is that possible to:

    1. define a logger interface to be used inside the server
    2. expose the error handle in above code (I didn't have time to go through the code, but I think we should be able to fix it to avoid panic)
  • FYI: my modifications to enable server use as a library

    FYI: my modifications to enable server use as a library

    Am not expecting you to merge this PR. This is mainly a FYI (don't know if there are better tools to "Show my change on top of upstream")

    EDIT: effort continues in new PR https://github.com/gokrazy/rsync/pull/11 & issue https://github.com/gokrazy/rsync/issues/12

  • client stuck on receiving filelist

    client stuck on receiving filelist

    I am trying out your implementation as described in the README.

    The Server is running successfully, but the client is stuck on:

    receiving file list ... done

    Here is the output from the server side.

    2021/08/23 08:13:47 rsync module "pwd" with path /tmp/test configured
    2021/08/23 08:13:47 rsync daemon listening on rsync://127.0.0.1:8730
    2021/08/23 08:14:08 client requested rsync module "pwd"
    2021/08/23 08:14:08 client sent: "--server"
    2021/08/23 08:14:08 client sent: "--sender"
    2021/08/23 08:14:08 client sent: "-vlogDtpr"
    2021/08/23 08:14:08 client sent: "."
    2021/08/23 08:14:08 client sent: "pwd/"
    2021/08/23 08:14:08 client sent: ""
    2021/08/23 08:14:08 flags: [--server --sender -vlogDtpr . pwd/]
    2021/08/23 08:14:08 remaining: ["." "pwd/"]
    2021/08/23 08:14:08 file list sent
    2021/08/23 08:14:08 exclusion list read
    
  • build_hash_table seems not point to the first entry

    build_hash_table seems not point to the first entry

    // “A 16 bit index table is then formed which takes a 16 bit hash // value and gives an index into the sorted signature table which // points to the first entry in the table which has a matching // hash.” for idx := range head.Sums { tagTable[targets[idx].tag] = idx }

    here the latter will overwrite the previous value

  • Fix some issues found by lint

    Fix some issues found by lint

    Issue: #10

    I fixed only what I think are "uncontroversial", i.e. I didn't yet mark any "lint ignore" annotations for more aggressive/"controversial" linters, because we don't yet have any agreed-on linter configuration.

    Each commit explains what I fixed.

    If you prefer, I can squash the commits before merging.

  • Remove module map from outside of rsyncd

    Remove module map from outside of rsyncd

    Bonus work found during https://github.com/gokrazy/rsync/issues/12

    (Extracted from previous PR: https://github.com/gokrazy/rsync/pull/11)

    (GitHub actions build pass in my fork)

  • Improve handling of dangling symlinks when creating file list

    Improve handling of dangling symlinks when creating file list

    Hey @stapelberg 👋

    Toying around with your tool/library here and ran into a small problem I'm a bit befuddled by: (I'm trying to see if I can build a container backup solution using rsync as the primary driver)

    I'm an invoking an rsync(1) client invoking the gokr-rsync over a shell like so:

    $ rsync --rsh="docker run -i --rm -v data:/data prologic/docker-rsync" --list-only :
    2022/12/30 23:44:31 remote protocol: 31
    2022/12/30 23:44:31 exclusion list read
    2022/12/30 23:44:31 sendFileList(module="implicit")
    2022/12/30 23:44:31   path "." (module root "/")
    2022/12/30 23:44:31 lstat /proc/1/fd/3: no such file or directory
    gokr-rsync [sender]: lstat /proc/1/fd/3: no such file or directory
    rsync: connection unexpectedly closed (71 bytes received so far) [Receiver]
    rsync error: error in rsync protocol data stream (code 12) at io.c(231) [Receiver=3.2.7]
    

    I'm not sure why /proc/1/fd/3 is being opened? I poked around in the container that is spawned and there is no file descriptor 3, only 0 1 2 as you'd expect (stdin, stdout, stderr).

    My Dockerfile is mostly similar to yours:

    FROM golang:alpine AS build
    
    RUN go install github.com/gokrazy/rsync/cmd/gokr-rsyncd@latest
    
    FROM alpine AS runtime
    
    COPY --from=build /go/bin/gokr-rsyncd /usr/local/bin
    
    #USER nobody:nobody
    
    VOLUME /data
    
    COPY entrypoint.sh /entrypoint
    
    ENTRYPOINT ["/entrypoint"]
    

    And the entrypoint.sh:

    #!/bin/sh
    
    cd /data || exit 1
    # exec gokr-rsyncd --daemon --gokr.listen=0.0.0.0:8730 --gokr.modulemap=pwd=$PWD
    exec gokr-rsyncd --server --sender . .
    
  • panic from hashSearch

    panic from hashSearch

    Hey,

    Background

    I am building an application that makes use of rsync as a file server to sync files to remote. In my implementation, it would potentially that the file contents changed during the file sync.

    Issue & Symptom

    I am seeing some random panics from the rsync call. Stack trace looks like this:

    panic: runtime error: index out of range [0] with length 0
    
    goroutine 733 [running]:
    github.com/gokrazy/rsync/rsyncd.(*sendTransfer).hashSearch(0x140005cfbe0, {0x140027f5788, 0x1, 0x28?}, 0x64?, {0x1, 0x2bc, 0x2, 0x2, {0x140030a9c20, ...}}, ...)
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/match.go:172 +0x9d0
    github.com/gokrazy/rsync/rsyncd.(*sendTransfer).sendFiles(0x140005cfbe0, 0x140033c5140)
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/sender.go:85 +0x320
    github.com/gokrazy/rsync/rsyncd.(*Server).HandleConn(0x1400209afc0, {{0x1036e4a34, 0x9}, {0x140000421e0, 0x2a}, {0x0, 0x0, 0x0}}, {0x103d7e2b8?, 0x140003ac0c0}, ...)
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/rsyncd.go:388 +0x3c8
    github.com/gokrazy/rsync/rsyncd.(*Server).HandleDaemonConn(0x1400209afc0, {0x14001000960?, 0x140010009c0?}, {0x12c50db18?, 0x14000011c18}, {0x103d886e0?, 0x1400212a660})
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/rsyncd.go:309 +0xb28
    github.com/gokrazy/rsync/rsyncd.(*Server).Serve.func2()
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/rsyncd.go:441 +0xb4
        created by github.com/gokrazy/rsync/rsyncd.(*Server).Serve
    <path>/go/pkg/mod/github.com/gokrazy/[email protected]/rsyncd/rsyncd.go:439 +0xc8
    

    I can reproduce this issue from the commit: https://github.com/gokrazy/rsync/commit/b567d9d1a3029ed39da65e3641471f058efa5d23 . But I am not yet reproduce this issue in commits after this.

    Unfortunately, I don't have a stable way to reproduce this issue (I am trying to automate the process, but no luck yet).

    My guess

    My guess is, this issue is due to in-flight file change + wrong read size settings (which fixed in this commit: https://github.com/gokrazy/rsync/commit/f19f7934a18784e347bc91b5811db187d8383f3d). Could you confirm the fix commit f19f793 is for this behavior?

    Further Asks

    I checked the code, the fileio.go is doing some log trackings (which are commented out), and it also exit for unexpected data state: https://github.com/gokrazy/rsync/blob/f3bbd0836728c03a03b1c26b86c040f277282ac7/rsyncd/fileio.go#L108-L111

    Is it possible to:

    1. record the state for the mapStruct during the ptr call to capture the value of offset, last readSize / readOffset etc
    2. panic instead of os.Exit(1) for these unexpected cases
    3. setup panic handler in the rsyncd server, then we can protect the server from panics like the one from this issue or from the mapStruct.ptr call

    What do you think?

  • Issues found by static analysis

    Issues found by static analysis

    Tasks:

    • [x] Send PR for most clear issues
    • [x] Have the above PR be merged
    • [ ] Agree on a lint rule set
    • [ ] Fix the rest of the issues found by agreed-on lint rule set
    • [ ] Set up GitHub action so there will be no new regressions

    I ran golangci-lint, with configuration I've settled on for my projects. Here's the current issues it found:

    Lint report
    pkg/rsyncd/rsyncd.go:76:6: `sumBuf` is unused (deadcode)
    type sumBuf struct {
         ^
    pkg/maincmd/unprivileged_linux.go:10:6: `runAsUnprivilegedUser` is unused (deadcode)
    func runAsUnprivilegedUser(cmd *exec.Cmd) {
         ^
    pkg/rsyncchecksum/rsyncchecksum.go:52:14: Error return value of `binary.Write` is not checked (errcheck)
    	binary.Write(h, binary.LittleEndian, seed)
    	            ^
    pkg/rsyncwire/wire.go:96:14: Error return value of `binary.Write` is not checked (errcheck)
    	binary.Write(&b.buf, binary.LittleEndian, data)
    	            ^
    pkg/rsyncwire/wire.go:100:14: Error return value of `binary.Write` is not checked (errcheck)
    	binary.Write(&b.buf, binary.LittleEndian, data)
    	            ^
    pkg/rsyncwire/wire.go:115:16: Error return value of `io.WriteString` is not checked (errcheck)
    	io.WriteString(&b.buf, data)
    	              ^
    pkg/anonssh/anonssh.go:166:14: Error return value of `req.Reply` is not checked (errcheck)
    				req.Reply(false, errmsg)
    				         ^
    pkg/anonssh/anonssh.go:167:18: Error return value of `channel.Write` is not checked (errcheck)
    				channel.Write(errmsg)
    				             ^
    pkg/anonssh/anonssh.go:180:17: Error return value of `newChan.Reject` is not checked (errcheck)
    		newChan.Reject(ssh.UnknownChannelType, fmt.Sprintf("unknown channel type: %q", t))
    		              ^
    pkg/receivermaincmd/receiver.go:98:19: Error return value of `out.Cleanup` is not checked (errcheck)
    	defer out.Cleanup()
    	                 ^
    pkg/rsyncd/rsyncd.go:198:17: Error return value of `io.WriteString` is not checked (errcheck)
    		io.WriteString(cwr, s.formatModuleList())
    		              ^
    pkg/rsyncd/rsyncd.go:199:17: Error return value of `io.WriteString` is not checked (errcheck)
    		io.WriteString(cwr, "@RSYNCD: EXIT\n")
    		              ^
    pkg/rsyncd/rsyncd.go:253:15: Error return value of `mpx.WriteMsg` is not checked (errcheck)
    		mpx.WriteMsg(rsyncwire.MsgError, []byte(fmt.Sprintf("gokr-rsync [sender]: %v\n", err)))
    		            ^
    pkg/rsyncd/rsyncd.go:312:16: Error return value of `mpx.WriteMsg` is not checked (errcheck)
    			mpx.WriteMsg(rsyncwire.MsgError, []byte(fmt.Sprintf("gokr-rsync [sender]: %v\n", err)))
    			            ^
    pkg/rsynctest/rsynctest.go:109:15: Error return value of `srv.Serve` is not checked (errcheck)
    		go srv.Serve(context.Background(), ts.listener)
    		            ^
    pkg/anonssh/anonssh.go:60:3: commentFormatting: put a space between `//` and comment text (gocritic)
    		//s.env = append(s.env, fmt.Sprintf("%s=%s", r.VariableName, r.VariableValue))
    		^
    pkg/receivermaincmd/receivermaincmd.go:169:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
    		} else {
    		       ^
    pkg/rsyncd/match.go:126:6: commentFormatting: put a space between `//` and comment text (gocritic)
    					//falseAlarms++
    					^
    pkg/rsyncd/match.go:181:3: commentFormatting: put a space between `//` and comment text (gocritic)
    		//log.Printf("null_tag, k=%d", k)
    		^
    cmd/gokr-rsyncd/rsyncd.go:25:3: exitAfterDefer: log.Fatal will exit, and `defer cancel()` will not run (gocritic)
    		log.Fatal(err)
    		^
    cmd/libuser/main.go:23:3: exitAfterDefer: log.Fatal will exit, and `defer cancel()` will not run (gocritic)
    		log.Fatal(err)
    		^
    pkg/rsyncchecksum/checksum_test.go:28:12: G306: Expect WriteFile permissions to be 0600 or less (gosec)
    	if err := ioutil.WriteFile(large, content, 0644); err != nil {
    	          ^
    pkg/receivermaincmd/receivermaincmd.go:277:9: G204: Subprocess launched with a potential tainted input or cmd arguments (gosec)
    	ssh := exec.Command(args[0], args[1:]...)
    	       ^
    pkg/maincmd/maincmd.go:18:2: G108: Profiling endpoint is automatically exposed on /debug/pprof (gosec)
    	_ "net/http/pprof"
    	^
    pkg/maincmd/writetest.go:12:12: G306: Expect WriteFile permissions to be 0600 or less (gosec)
    	if err := ioutil.WriteFile(fn, []byte("gokr-rsyncd creates this file to prevent misconfigurations. if you see this file, it means gokr-rsyncd unexpectedly was started with too many privileges"), 0644); err == nil {
    	          ^
    pkg/rsynctest/rsynctest.go:251:12: G306: Expect WriteFile permissions to be 0600 or less (gosec)
    	if err := ioutil.WriteFile(large, content, 0644); err != nil {
    	          ^
    pkg/anonssh/anonssh.go:43:2: `env` is unused (structcheck)
    	env     []string
    	^
    pkg/anonssh/anonssh.go:44:2: `ptyf` is unused (structcheck)
    	ptyf    *os.File
    	^
    pkg/anonssh/anonssh.go:45:2: `ttyf` is unused (structcheck)
    	ttyf    *os.File
    	^
    pkg/anonssh/anonssh.go:27:2: `cfg` is unused (structcheck)
    	cfg  *config.Config
    	^
    pkg/rsyncd/rsyncd.go:80:2: `sum1` is unused (structcheck)
    	sum1   uint32
    	^
    pkg/rsyncd/rsyncd.go:81:2: `sum2` is unused (structcheck)
    	sum2   [16]byte
    	^
    pkg/rsyncd/rsyncd.go:77:2: `offset` is unused (structcheck)
    	offset int64
    	^
    pkg/rsyncd/rsyncd.go:78:2: `len` is unused (structcheck)
    	len    int64
    	^
    pkg/rsyncd/rsyncd.go:79:2: `index` is unused (structcheck)
    	index  int32
    	^
    pkg/rsyncd/token.go:39:36: unnecessary conversion (unconvert)
    		return st.conn.WriteInt32(-(int32(token) + 1))
    		                                 ^
    pkg/anonssh/anonssh.go:50:27: `(*session).request` - `ctx` is unused (unparam)
    func (s *session) request(ctx context.Context, req *ssh.Request) error {
                              ^
    pkg/receivermaincmd/generator.go:46:66: (*recvTransfer).skipFile - result 1 (error) is always nil (unparam)
    func (rt *recvTransfer) skipFile(f *file, st os.FileInfo) (bool, error) {
                                                                     ^
    pkg/receivermaincmd/receivermaincmd.go:239:12: `doCmd` - `osenv` is unused (unparam)
    func doCmd(osenv osenv, opts *Opts, machine, user, path string, daemonConnection int) (io.ReadCloser, io.WriteCloser, error) {
               ^
    pkg/rsyncd/flist.go:17:38: `(*sendTransfer).sendFileList` - `c` is unused (unparam)
    func (st *sendTransfer) sendFileList(c *rsyncwire.Conn, mod config.Module, opts *Opts, paths []string) (*fileList, error) {
                                         ^
    pkg/rsyncd/token.go:45:79: `(*sendTransfer).sendToken` - `toklen` is unused (unparam)
    func (st *sendTransfer) sendToken(f *os.File, i int32, offset int64, n int64, toklen int64) error {
                                                                                  ^
    pkg/rsyncd/match.go:160:5: unreachable: unreachable code (govet)
    				offset += head.Sums[i].Len - 1
    				^
    pkg/rsyncd/match.go:182:3: unreachable: unreachable code (govet)
    		readk := k + 1
    		^
    pkg/rsyncchecksum/rsyncchecksum.go:26:2: variable len has same name as predeclared identifier (predeclared)
    	len := len(buf)
    	^
    pkg/rsynccommon/rsynccommon.go:14:21: param len has same name as predeclared identifier (predeclared)
    func SumSizesSqroot(len int64) rsync.SumHead {
                        ^
    pkg/receivermaincmd/generator.go:263:58: param len has same name as predeclared identifier (predeclared)
    func (rt *recvTransfer) generateAndSendSums(in *os.File, len int64) error {
                                                             ^
    pkg/receivermaincmd/receiver.go:124:3: variable len has same name as predeclared identifier (predeclared)
    		len := sh.BlockLength
    		^
    pkg/maincmd/maincmd.go:44:3: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
    		fmt.Fprintf(stderr, opt.Help())
    		^
    pkg/receivermaincmd/generator.go:182:2: SA9003: empty branch (staticcheck)
    	if rt.opts.PreserveHardlinks {
    	^
    pkg/receivermaincmd/generator.go:89:2: SA4006: this value of `st` is never used (staticcheck)
    	st, err = rt.setUid(f, local, st)
    	^
    

    Some of these are non-issues, but some seem clear bugs like this one:

    pkg/maincmd/maincmd.go:44:3: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck)
    		fmt.Fprintf(stderr, opt.Help())
    

    Would you like me to submit a PR to fix all or some of these complaints?

  • Implement an rsync receiver, too

    Implement an rsync receiver, too

    Currently, we only implement a sender (for serving files), but a receiver would be neat, too (for downloading files).

    Left to do:

    • [x] implement receiver protocol
    • [x] write files to disk
    • [x] only transfer deltas
    • [ ] cross-system user/group id/name mapping
  • Use fs.FS for filesystem access

    Use fs.FS for filesystem access

    Go's new FS interface is pretty cool, I've used in quite a few projects to serve files embedded into the binary, to do overlayFS-like stuff (merge multiple directories as one) etc. fs.FS enables all of this craziness.

    My proposal: instead of giving direct filesystem path like /my/path to a module, the module would take in fs.FS and you could give in os.DirFS("/my/path").

    I am willing to send a PR if you are interested. I have not read through the server code to check if there are any requirements that don't have direct analogies in fs interfaces. uid/gid things in Stat() calls would be one thing not present in fs.FileInfo. Those cases would need to be type-switched on (check if stat result is a structure that actually carries uid/gid)

The modules is contains the golang utilities for internal services

Shared Utility The modules is contains the golang utilities for internal service

Jan 25, 2022
This repo contains a sample app exposing a gRPC health endpoint to demo Kubernetes gRPC probes.

This repo contains a sample app exposing a health endpoint by implementing grpc_health_v1. Usecase is to demo the gRPC readiness and liveness probes introduced in Kubernetes 1.23.

Feb 9, 2022
go implementation of fissions web-native file system

wnfs-go go language implementation of the fission web-native file system, using the typescript implementation as a reference. Development Status: Work

Oct 15, 2022
A native Thrift package for Go

Thrift Package for Go API Documentation: http://godoc.org/github.com/samuel/go-thrift License 3-clause BSD. See LICENSE file. Overview Thrift is an ID

Nov 22, 2022
A go implementation of global-countries package

global-countries-gosdk A Golang implementation of global-countries Installation go get github.com/GoodnessEzeokafor/global-countries-go // get all cou

Oct 4, 2021
Ratelimit - This package provides a Golang implementation of the leaky-bucket rate limit algorithm

Go rate limiter This package provides a Golang implementation of the leaky-bucke

Jul 26, 2022
A cloud native distributed streaming network telemetry.
A cloud native distributed streaming network telemetry.

Panoptes Streaming Panoptes Streaming is a cloud native distributed streaming network telemetry. It can be installed as a single binary or clustered n

Sep 27, 2022
A simple TUN/TAP library written in native Go.

water water is a native Go library for TUN/TAP interfaces. water is designed to be simple and efficient. It wraps almost only syscalls and uses only G

Jan 7, 2023
Golang client for NATS, the cloud native messaging system.

NATS - Go Client A Go client for the NATS messaging system. Installation # Go client go get github.com/nats-io/nats.go/ # Server go get github.com/na

Jan 4, 2023
The Cloud Native Application Proxy
The Cloud Native Application Proxy

Traefik (pronounced traffic) is a modern HTTP reverse proxy and load balancer that makes deploying microservices easy. Traefik integrates with your ex

Dec 30, 2022
🤘 The native golang ssh client to execute your commands over ssh connection. 🚀🚀
🤘 The native golang ssh client to execute your commands over ssh connection. 🚀🚀

Golang SSH Client. Fast and easy golang ssh client module. Goph is a lightweight Go SSH client focusing on simplicity! Installation ❘ Features ❘ Usage

Dec 24, 2022
grobotstxt is a native Go port of Google's robots.txt parser and matcher library.

grobotstxt grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port Preserve

Dec 27, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

中文 MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

Dec 30, 2022
Cloud Native Tunnel
Cloud Native Tunnel

inlets is a Cloud Native Tunnel written in Go Expose your local endpoints to the Internet or within a remote network, without touching firewalls. Foll

Jan 4, 2022
Native ZooKeeper client for Go. This project is no longer maintained. Please use https://github.com/go-zookeeper/zk instead.

Native Go Zookeeper Client Library License 3-clause BSD. See LICENSE file. This Repository is No Longer Maintained Please use https://github.com/go-zo

Dec 19, 2022
Native macOS networking for QEMU using vmnet.framework and socket networking.

qemu-vmnet Native macOS networking for QEMU using vmnet.framework and socket networking. Getting started TODO -netdev socket,id=net0,udp=:1234,localad

Jan 5, 2023
zMemif is a native golang based library for memif to interworking with dpdk.

zMemif zMemif is a native golang based library for memif to interworking with dpdk. it can simply provide 20Mpps recv and 10Mpps xmit capability. The

Dec 27, 2022
Fetch-npm-package - A small utility that can be used to fetch a given version of a NPM package

Use fetch-npm-package <package> <version> <output-dir> E.g. fetch-npm-package is

May 21, 2022
A go implementation of the STUN client (RFC 3489 and RFC 5389)

go-stun go-stun is a STUN (RFC 3489, 5389) client implementation in golang (a.k.a. UDP hole punching). RFC 3489: STUN - Simple Traversal of User Datag

Jan 5, 2023