With this PR transfer.sh has native bindings to the Storj network. Previously it was using the S3 backend and proxying the data through a S3 Gateway ( https://github.com/storj/storj/cmd/gateway/ ).

@aspacca If needed, i can grant access to the Beta Network to try it out. I would appreciate a throughout test!

doing this curl --upload-file quiet_01.txt https://transfer.sh and it returns curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)

I have a private transfer.sh instance running with authentication switched on. The setup is as follows:

1. transfer.sh running on http://localhost:4000 on Centos 7
2. Exposed to the internet via nginx with http redirect to https
3. X-Forwarded-Proto is set so that transfer.sh displays all urls as https

Issue:

1. When I upload small files of a few KB the browser pop up for auth appears and accepts the user id and password.
2. When I upload larger files (of a few MB) that take a few seconds to upload, the authentication popup never appears and after a few seconds after reaching 100% a (502) error is displayed like this: Error (502) during upload of file .....
3. In the transfer.sh server log a 401 error is recorded
4. The workaround is to first upload a small file, authenticate and then upload the large file and it then works.

Any ideas if this is a config/landscape fix or needs a code change?

I can't upload anything on your server, it makes the speed very slow(sometimes directly 0)

I also requested IP change from my host and new IP also having same issue. This is 4th time in this week(I upload once every day). I also know what you guys want from me so there you go:

Running applications in Docker as root user opens various attack vectors, as Docker's "isolation" isn't meant to be the only safe guard against adversaries. It should be considered a "last resort". It's best practice to use a non-root user wherever possible.

This commit changes back to Alpine as a base, as it provides the adduser tool via busybox. It increases the image size by a few MiB (2-3, currently). One could optimize the image size by writing a custom /etc/passwd, but this seems quite unnecessary, especially given that Alpine is the base of so many other images.

Fixes #100.

I have read #22. What about allowing creating accounts, and if a file is uploaded in an account, then permit raw access? If GitHub account should be used for transfer.sh account creation, unethical use of the service could be lowered.

By setting u.MaxUploadParts = concurrency and u.PartSize = (1 << 20) * 5, there was a 100MB limit on S3 uploads. This PR removes that limit leaving MaxUploadParts at the default of 10,000 and allowing the option to set the part size for S3 uploads with s3-part-size.

Hi @stefanbenten ! I read through implementation discussion here: https://github.com/dutchcoders/transfer.sh/issues/44#issuecomment-399234697. So I understand it is not an issue with the code. It is the the website transfer.sh that should provide a delete form to process the delete GET-request, correct? So my wish would be that such a page would be set up for transfer.sh so that deletes can be successfully performed..

I'd also appreciate a reconsideration of that discussion. In my view a direct delete by a GET request (without DELETE header) should be made possible without the need for an additional form or button. To prevent accidental or malicious deletes via crawler the delete link should be constructed more randomly with longer passwords and should be different from the download links and not derived from them. I believe atm they are too similar and can be easily computed if the download link is known.

Originally posted by @beriner in https://github.com/dutchcoders/transfer.sh/issues/351#issuecomment-783746108

@bookmoons this is a skeleton of what I think should look like the fuzz test please review, and if it's ok I will properly randomize token and filename and all the rest

alternatively instead of testing directly the storage we can build the http request to the listener that will hit the equivalent method in the storage that I tested now: what would be the best solution?

btw: I didn't run the code yet

Hello,

I just want to share this. If you consider my function as an worthy example for the homepage, go for it!:

transfer()
{
    local file="${1}"
    local filename="${file##*/}"
    # show delete link from the response header after upload. the command "sed" is necessary to clean up the output, "gsub()" in "awk" does not work.
    curl --request PUT --progress-bar --dump-header - --upload-file "${file}" "https://transfer.sh/${filename}" | sed "s/#//g" | awk '/x-url-delete/ { print "Delete command: curl --request DELETE " $2 } END{ print "Download link: " $1 }'
}

Example outputs:

$ transfer image.img
######################################################################################################################################################################################################################################## 100.0%
Delete command: curl --request DELETE https://transfer.sh/Ge9cuW/image.img/<some_delete_token>
Download link: https://transfer.sh/Ge9cuW/image.img

$ transfer /tmp/tmp.LkwBEu5r64/image2.img
######################################################################################################################################################################################################################################## 100.0%
Delete command: curl --request DELETE https://transfer.sh/oeopqy/image2.img/<some_delete_token>
Download link: https://transfer.sh/oeopqy/image2.img

-Ramon

Hi there! Thanks for making and hosting such a useful service!

It looks like there's some issue with the backend right now?

$ curl -v https://transfer.sh/
*   Trying 54.77.221.1...
* Connected to transfer.sh (54.77.221.1) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: transfer.sh
* Server certificate: COMODO RSA Domain Validation Secure Server CA
* Server certificate: COMODO RSA Certification Authority
* Server certificate: AddTrust External CA Root
> GET / HTTP/1.1
> Host: transfer.sh
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable: Back-end server is at capacity
< Content-Length: 0
< Connection: keep-alive
<
* Connection #0 to host transfer.sh left intact

I have setup storj as provider. I can successfully upload the file, (I can verify the uploaded file in the storj bucket) but when I open the output url it giving me error 'Not found'. Log

 [transfer.sh]2022/12/28 06:16:09 Error metadata: EOF
 [transfer.sh]2022/12/28 06:16:08 Getting file 62bc21edb1deb.mp4.metadata from Storj Bucket
[transfer.sh]2022/12/28 06:16:04 47.31.141.30:0 - - [2022-12-28T06:16:04Z] "PUT /62bc21edb1deb.mp4 HTTP/1.1" 200  "https://*******.sh/"; "Mozilla/5.0 (Android 11; Mobile; rv:108.0) Gecko/108.0 Firefox/108.0"
 [transfer.sh]2022/12/28 06:15:51 Uploading file 62bc21edb1deb.mp4 to Storj Bucket
 [transfer.sh]2022/12/28 06:15:51 Uploading Bdm5IY 62bc21edb1deb.mp4 11009799 
 [transfer.sh]2022/12/28 06:15:51 Uploading file 62bc21edb1deb.mp4.metadata to Storj Bucket

I am using docker with local provider and activated http basic auth. But as long as you know the delete url, you can delete the file immediately without providing username and password, is it intended?

I did a search and found it might be a size limit issue, then I looked at the README, which is extremely confusing.

How should I pass those "parameters" to transfer.sh or its docker container? According to "usage" table it looks like I should use environment variables but passing MAX_UPLOAD_SIZE didn't work.

Relates to #475, #478 and #483.

This improves the function transfer() to be able to upload files with spaces in their filenames.

Every space character is replaced with the URL encoding character: %20.

I also added some more examples.

$ transfer --help
[...]

EXAMPLES:
[...]
  Upload multiple files with spaces in their filename from the current working directory:
      transfer "image 1.img" image\ 2.img
[...]
  Upload multiple files from a different directory:
      transfer transfer /tmp/{image1.img,image\ 1.img,"image 2.img"}
[...]

$ transfer "image 1.img" image\ 2.img
4K      image 1.img
4K      image 2.img
8K      total
Do you really want to upload the above files (2) to "transfer.sh"? (Y/n):
######################################################################################################################################################################################################################################## 100.0%
Delete command: curl --request DELETE "https://transfer.sh/V4X6y0/image%201.img/gBjLz3jMRxd4"
Delete token: gBjLz3jMRxd4
Download link: https://transfer.sh/V4X6y0/image%201.img

######################################################################################################################################################################################################################################## 100.0%
Delete command: curl --request DELETE "https://transfer.sh/CHLy0j/image%202.img/hPd1EfqSbCk7"
Delete token: hPd1EfqSbCk7
Download link: https://transfer.sh/CHLy0j/image%202.img

image 1.img | image 2.img :----------------: | :-----------------: |

For personal use, I currently working on adding support for Azure blob storage as an alternative storage driver (https://github.com/joshspicer/transfer.sh/blob/main/server/storage/azure.go).

I would love to contribute back the Azure storage driver back to this repo. The Azure SDKs require at least go 1.18. I am curious if there's any justification for pinning this project to go 1.15, which is now unsupported.