Occasionally (about once every 12-18 hours) the server will stop accepting the correct username and password. It requires a service reboot to fix. This is the closest set of logs to the latest attempt to login while in this error state:

{"level":"debug","sender":"sftpd","connection_id":"","time":"2019-09-11T03:27.05.832","message":"idle connections check ticker 2019-09-11 03:27:05.832582724 +0000 UTC m=+44100.001673267"}
{"level":"info","sender":"sftpd","connection_id":"6508f97ceb9b67f8e7fc4928cd9b0cd10615a47024a8279611e9e8f3f6ef24b4","time":"2019-09-11T03:27.05.832","message":"close idle connection, idle time: 6h27m4.548987039s"}
{"level":"warn","sender":"sftpd","connection_id":"6508f97ceb9b67f8e7fc4928cd9b0cd10615a47024a8279611e9e8f3f6ef24b4","time":"2019-09-11T03:27.05.832","message":"idle connection close failed: close tcp 172.31.18.230:22->148.59.44.16:64168: use of closed network connection"}
{"level":"info","sender":"sftpd","connection_id":"8fe250f6489ada67fcb4161871743f4358892324ce8841dcbe4903eabfcc2a59","time":"2019-09-11T03:27.05.832","message":"close idle connection, idle time: 6h27m4.535060735s"}
{"level":"warn","sender":"sftpd","connection_id":"8fe250f6489ada67fcb4161871743f4358892324ce8841dcbe4903eabfcc2a59","time":"2019-09-11T03:27.05.832","message":"idle connection close failed: close tcp 172.31.18.230:22->148.59.44.16:64169: use of closed network connection"}
{"level":"debug","sender":"sftpd","connection_id":"","time":"2019-09-11T03:27.05.832","message":"check idle connections ended"}

Any ideas? Is there any way to stop it trying to close idle connections? Perhaps that's the problem?

Hi, Thanks for this great project !

I did some test in my environment and the transfer speed is much lower than OpenSSH.

Server || --- | --- | OS| Debian 10.2 x64 | CPU| Ryzen5 3600 | RAM| 64GB ECC | Disk| 3* Intel P4510 4TB RAID0 | Ethernet| Mellanox ConnectX-3 40GbE|

Client || --- | --- | OS| Windows 10 1909 x64 | CPU| Threadripper 1920X | RAM| 64GB ECC | Disk| Samsung 960EVO 1TB | Ethernet| Mellanox ConnectX-3 40GbE|

Under Filezilla I can get 500MB/s with OpenSSH, but only about 200MB/s with sftpgo.

In both case I'm using AES256-CTR as cipher and SHA-256 as MAC, I've also tried AES128-CTR but nothing changes.

CPU usage of sftpgo is higher than OpenSSH:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND 
 4527 sftp      20   0 1795576  52044   8628 R 133.5   0.6   2:12.13 sftpgo 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND 
27934 xxxxxx    20   0   17112   5360   4188 R  67.8   0.1   0:10.01 sshd                                                  
27942 xxxxxx    20   0   17112   5344   4176 R  27.4   0.1   0:12.52 sshd 

In both case I've got a maximum TCP window size of 4MB.

I have a strange issue with an account:

My user defitinion is as follows:

      {
        "id": 1,
        "status": 1,
        "username": "test",
        "password": "***",
        "home_dir": "/sftp",
        "uid": 1100,
        "gid": 1100,
        "permissions": {
          "/": [
            "list"
          ],
          "/sftp/test/upload: [
            "list",
            "upload",
            "download",
            "rename",
            "create_dirs",
            "delete"
          ]
        }
      },

Although I can list in all directories in /sftp, I wanted to be able to upload to /sftp/test/upload with the above definition. Am i doing something incorrect here?

Using edge

This PR adds a Dockerfile to the project. It's largely based on my previous Go applications and the existing examples in the repo. It's not perfect though, so we might need some fine tuning in the future, particularly around volumes and configuration.

Open questions:

• What are backups? Are they supposed to be saved to persistent storage or are they only meaningful during the lifecycle of the application?
• Where does the credential path point? What's being read from there? (Existing dockerfiles contain no reference)

Currently the above settings can only be overridden from environment variables, because defaults are defined for them in env vars. (I think that's an acceptable compromise though).

I'll work with this image anyway, so I'll provide patches along the way.

The PR also adds a GitHub Actions workflow for building Docker images based on the official example (I removed a few parts):

https://github.com/docker/build-push-action#complete-workflow

After merging the PR, @drakkan , you'll need to do some manual steps though:

1. Login to ghcr.io: docker login ghcr.io. You'll need a GH PAT for that (with package write scope)
2. Build the image locally: docker build -t ghcr.io/drakkan/sftpgo:edge .
3. Push the image docker push ghcr.io/drakkan/sftpgo:edge
4. Go to packages under your account
5. Make the package public
6. Configure a secret called CR_PAT with package write scope under secrets
7. Uncomment the relevant parts in docker.yml (login and push) (happy to review a PR)

These steps are necessary, because GitHub packages seem to be super private for the first time, so you need to push and publish manually. (Although rereading the above: you should also be fine to do step 6 and 7 first, since CR_PAT will be your token anyway, and then step 5 to publish the package. I had some trouble with packages under organizations, but this is under your personal account, so it should be fine)

Sorry about the complicated process, let me know if you need any help.

Hello, I would like to install SftpGO with Traefik, the problem is that it does not work,

I access the dashboard without problem but when I connect with Filezilla in SFTP it doesn't work and I really don't understand why...?

My configuration :

traefik.yml

docker-compose.yml

`version: '3.3' services:

ftp: container_name: ftp image: drakkan/sftpgo:edge-alpine-slim networks: network-traefik: ipv4_address: 192.168.128.xxx labels: - "traefik.enable=true" - "traefik.tcp.routers.ftp.rule=HostSNI(*)" - "traefik.tcp.routers.ftp.entrypoints=sftp" - "traefik.tcp.services.ftp.loadbalancer.server.port=2022" - "traefik.http.routers.uiftp.rule=Host(MYDOMAIN.FR)" - "traefik.http.routers.uiftp.entrypoints=https" - "traefik.http.routers.uiftp.tls.certresolver=http" - "traefik.http.services.uiftp.loadbalancer.server.port=8080"

networks: network-traefik: external: true`

I'm really stuck help me! :D

Hey there @drakkan, thank so much for your work on this, it's a great tool!

I'm trying to connect to an SFTP server on the backend (server using OpenSSH) but am getting error SSH_FX_FAILURE when doing so. It only affects WebDAV. Without changing any settings, if I open the virtual folder using an SFTP connection, it shows the backend server's file structure fine.

Computer via SFTP > SFTPGo > Virtual Folder SFTP Backend > Success Computer via WebDAV > SFTPGo > Virtual Folder SFTP Backend > Fails with SSH_FX_FAILURE

Figured out what is causing this. If a folder is symlinked, SFTP doesn't find out about it until that folder is opened, whereas WebDAV, at least in Windows, tries to access that folder upon opening its parent folder.

Computer via SFTP > SFTPGo > Virtual Folder SFTP Backend: accessing /parent with /parent/inaccessible subdirectory inside of it works until I open /parent/inaccessible any accessible folders are fine

Computer via WebDAV > SFTPGo > Virtual Folder SFTP Backend: accessing /parent with /parent/inaccessible subdirectory inside of it does not let me open /parent at all cannot access any folders as long as at least 1 folder is inaccessible

Error from log: Oct 16 00:07:35 control sftpgo[29746]: {"level":"info","time":"2021-10-16T00:07:35.724","sender":"webdavd","elapsed_ms":86,"method":"PROPFIND","proto":"HTTP/2.0","remote_addr":"<redacted>","request_id":"c5l5s5p6q9grffh6gui0","uri":"https://mySFTPGOserver/virtualfolder","user_agent":"Microsoft-WebDAV-MiniRedir/10.0.19043","error":"sftp: \"Failure\" (SSH_FX_FAILURE)"}

Oct 16 00:43:43 control sftpgo[31376]: {"level":"warn","time":"2021-10-16T00:43:43.931","sender":"sftpfs \"<redacted>"","connection_id":"DAV_c5l6d3p6q9gri73uier0","message":"Invalid path resolution, dir \"/real/path/to/404\" original path \"/website/public_html/404\" resolved \"/home/user/web/website/public_html/404\" err: path \"/real/path/to/404\" is not inside: \"/home/user/web\""}

Also discovered another issue. If you're using WebDAV -> SFTPGo -> S3 Backend, a password of Abcde!+ will not work but Abcde will work. If using Abcde!+ logs say "invalid credentials". If you remove the S3 Backend virtual folder, without changing anything, you can successfully login without error (local folders work fine).

Thank you

It's a common practice to make packages that are supposed to be consumed by third party code (eg. plugins) separate modules. (Example: api packages in most Hashicorp software).

This would make the SDK package lighter in terms of dependencies (it doesn't need all deps of SFTPGo).

At the same time, until plugins are experimental, it might be easier to keep it in the same module (although making an existing package a module later has it's own quirks).

I am running a server on a local dir:

sftpgo portable -d ~/Base/shared --permissions 'list,download' --username 'x' --password 'x' --webdav-port 8114 --sftpd-port 8115 --ftpd-port 8116 --log-verbose --log-file-path sftpgo.log

I symlink other directories I want to share into ~/Base/shared, but sftpgo does not handle the symlinks correctly. (I know sftpgo supports virtual dirs, but I use that ~/Base/shared for other services, too, and I do not want to duplicate my config.)

PS: That command also does not start the webdav server, saying

{"level":"debug","time":"2021-03-05T10:49:41.514","sender":"service","message":"WebDAV server not started, disabled in config file"}

I configured the passive port range, but first forgot, then misplaced the passive ip configuration. Sftpgo then gives a confusing "index out of range" error, probably because it's searching for the passive ip and not finding it?

Currently configuring SFTPGo is a bit complicated in the various environments it might run in (thanks to the available Docker image).

Namely, the fact that the config dir is tied to a bunch of other things (host keys, sqlite, etc) and that you cannot pass a full path to a config file is a bit uncomfortable.

To give you a concrete example: when running on Kubernetes, I can mount a config map to a certain path (eg. /etc/sftpgo), containing an sftpgo.yaml file. The problem is, by default, SFTPGo will try to write default host keys to the config dir, but that mount is read only. Right now I solve this by manually setting the path to host keys.

I see two potential improvements:

• allow passing a full path to SFTPGo as a flag (analogous to viper.SetConfigFile)
• introduce a separate directory for host keys and other stuff (although there are already enough directories)

The first approach would allow mounting a config map under something like /etc/sftpgo/config (and since --config-file is already taken it would allow passing --config-path /etc/sftpgo/config/sftpgo.yaml) and would not interfere with the current config dir concept (/etc/sftpgo would still be writable).

hello, I use sftpgo as a bridge to an s3 endpoint that when a file is uploaded moves it elsewhere. Using webdav and uploading a file with windows the explorer say:

this happens because the file is no longer avilable and I guess it is done a stat that fails. Using Curl,for obvious reasons, is ok :

> PUT /fake0/out/test.bin HTTP/1.1
> Authorization: Basic XYZ
> User-Agent: curl/7.29.0
> Host: FQDN:2080
> Accept: */*
> Content-Length: 441
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 201 Created
< Etag: "1641d9285d8c3d961b9"
< Date: Tue, 27 Oct 2020 12:28:29 GMT
< Content-Length: 7
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host FQDN left intact

could it be a cache problem (assuming you use it) of the driver that uses sftpgo?

thnx a lot.

Sir, thank you very much for developing the tool, I read the document carefully, admin/client page external login only support OpenId connetion, but our internal authentication is auth2, can you give me some tips? thank you.

Hi!

I'm trying to investigate an issue where we are seeing two 'upload' actions get dispatched for the same file. The payload of the request contains identical data - including the same session Id.

For this file, we only got 1 'pre-upload' action. The payload in both upload actions indicate that the file was successfully uploaded with the correct file size.

I seem to notice this issue more when the system is under load via JMeter. We are using the default DB (SQLite) and I have seen errors related to SQLite 'context deadline exceeded'. We have considered moving to either the in-memory or Bolt DB. I'm not sure if the duplicate hooks getting dispatched could be related to the DB being under extreme load - perhaps triggering a retry of the upload hook.

Let me know what I can do to help :)

Thanks!

Edit: System Info Running on Azure "Premium SSD"

Hi,

I added the YUM repo and encountered an error when I tried to install sftpgo by running 'rpm-ostree install sftpgo'.

error: Importing RPMs: Importing package 'sftpgo' : Analyzing /srv/sftpgo: Unsupported path; see https://github.com/projectatomic/rpm-ostree/issues/233

Great software btw.

Hi,

I am using sftpgo with a S3 backend for storage. The user mounts the bucket on "/". As the bucket contains contains quite a few files (~8k) at the root. It takes a second to load but no issue there. Everything is listed and quite fast.

However, if I try to select a file (or all), the whole page becomes frozen and I have to basically close the tab and restart from another one. The behavior is the same in Safari, Firefox and Chrome.

With a single file in the bucket I don't get the issue, so I suppose selecting even a single element is doing something is the background that does not scales to 8k elements. Not a big issue for me but I wanted to report.