文件服务器的扩容是怎样实现的呢？比如现在已经配置了三个peer节点，这三个节点对等，会自动同步文件，实现了文件服务器的高可用。但发现有一天文件服务器磁盘快满了，我应该怎样扩容呢？ 扩容方案： 1）增加服务器，新增加的服务器跟现有peer节点之间数据不再同步，而是跟现有peer节点间形成2个分片式集群。也就是可以形成多个分片，每个分片数据都不相同，每个分片都可以有多个对等节点，形成高可用。这种实现方式有点类似于solr。 2）在现有节点上挂载新空间。利用存量对等节点，给每个节点都分配新的存储空间。这种方式有点类似于TFS。

经测试： auth_url不配置的时候，组内的两台机都能删除。 auth_url配置了，有一台不能删除，fileserver.log报错如下：

2021-04-30 11:37:34 [ERR] [fileserver.go:1009] [main.(*Server).CheckAuth] main.JsonResult.Status: ReadString: expects " or n, but found 4, error found in #10 byte of ...|"status":400,"error"|..., bigger context ...|mestamp":"2021-04-30T03:37:34.122+0000","status":400,"error":"Bad Request","message":"Required Strin|...

请问解决方法，谢谢～～

Hello,

I am looking for a good alternative to IPFS in Golang for a P2P project that would need to support many thousands of Peers and I am wondering how easily go-fastdfs scales if nodes are dynamically added and removed as in a P2P operation.

Also, it is possible to "pin" files to a specific node and still be accessible to those files from other nodes?

Any help would be greatly appreciated. Thanks in advance

按照文档上的代码使用hutool进行http上传，但是给我返回这个提示 request Content-Type isn't multipart/form-data

下面是代码： ` byte[] bytes = FileUtil.readBytes(filePath); File file = new File(filePath);

    InputStream inputStream = new ByteArrayInputStream(bytes);
    InputStreamResource isr = new InputStreamResource(inputStream, file.getName());
   

    //声明参数集合
    HashMap<String, Object> paramMap = new HashMap<String, Object>();
    //文件
    paramMap.put("file", isr);
    //输出
    paramMap.put("output", "json");
    //自定义路径
    //paramMap.put("path", "image");
    //场景
    paramMap.put("scene", "register");
    //上传
    String result = HttpUtil.post(MessageFormat.format("{0}/upload", address), paramMap);`

使用okhttp3进行上传（代码复制的文档中的示例代码）时，有时候正常，有时候提示

(String)"(error) if you want to upload fast md5 is require,and if you want to upload file,you must use post method

想问下这个情况的大概原因

当跨域使用并且打开auth_token验证的时候，某些情况浏览器会发送options请求，参考： https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Access_control_CORS#Preflighted_requests

但这个options请求进入upload方法后，会触发验证auth_token，但这时候是没有参数的，所以导致验证出错。fileserver.go(2141行)是不是要先判断一下请求类型，再调用验证？

在使用TUS断点上传时返回地址没有端口

位置TusClient.createUpload
String urlStr = connection.getHeaderField("Location");

docker-compose.yml

version: "3.7"

services:
  fastdfs0:
    container_name: fastdfs0
    image: sjqzhang/go-fastdfs
    volumes:
      - ./data0:/data
    environment:
      GO_FASTDFS_DIR: /data
    networks:
      fastdfs:
        ipv4_address: 172.30.0.2

  fastdfs1:
    container_name: fastdfs1
    image: sjqzhang/go-fastdfs
    volumes:
      - ./data1:/data
    environment:
      GO_FASTDFS_DIR: /data
    networks:
      fastdfs:
        ipv4_address: 172.30.0.3

  fastdfs2:
    container_name: fastdfs2
    image: sjqzhang/go-fastdfs
    volumes:
      - ./data2:/data
    environment:
      GO_FASTDFS_DIR: /data
    networks:
      fastdfs:
        ipv4_address: 172.30.0.4

  fastdfs-nginx:
    container_name: fastdfs-nginx
    image: nginx
    volumes:
     - ./nginx:/etc/nginx/conf.d
    depends_on:
     - fastdfs0
     - fastdfs1
     - fastdfs2
    networks:
      fastdfs:
        ipv4_address: 172.30.0.250
    ports:
     - "8080:80"

networks:
  fastdfs:
     driver: bridge
     ipam:
       driver: default
       config:
         - subnet: 172.30.0.0/16

nginx default.conf

upstream group1{
  server 172.30.0.2:8080;
  server 172.30.0.3:8080;
  server 172.30.0.4:8080;
  ip_hash;
}

log_format  upstream  '$remote_addr - $remote_user [$time_local] "$request" '
            '$status $body_bytes_sent "$http_referer" '
            '"$http_user_agent" "$http_x_forwarded_for" '
            '$upstream_addr $request_time $upstream_response_time ';

client_max_body_size 100M;

server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    access_log  /var/log/nginx/access.log  upstream;

    location /{
      proxy_set_header    Host $host;
      proxy_set_header    X-Real-IP $remote_addr;
      proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

      proxy_pass http://group1;
    }

    location /group1{
      proxy_set_header    Host $host;
      proxy_set_header    X-Real-IP $remote_addr;
      proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;

      proxy_pass http://group1;
    }

    #location / {
    #    root   /usr/share/nginx/html;
    #    index  index.html index.htm;
    #}

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

测试时候发现不是必然发生，但是概率不低。 就是多次上传相同文件，按照去重，应该返回的都是第一次上传的文件信息，但是有时候却返回的是无法打开文件失败，路径指向的是最新的地址。 open files/video/20190226/18/08/8/test.mp4: no such file or directory

使用的是sha1去重，搭建两台服务器，使用nginx做负载均衡。

在CentOS7中，首次启动报错如下

panic: open file path: conf/cfg.json error: open conf/cfg.json: permission denied


goroutine 1 [running]:
main.ParseConfig(0x99d8f6, 0xd)
	/root/fileserver.go:422 +0x76b
main.init.0()
	/root/fileserver.go:2801 +0x222
main.init()
	<autogenerated>:1 +0x12b

启动用户为普通账户，非root账户

解决方法也简单，首次启动时，生成的目录给属主有x权限即可，即

chmod u+x conf
chmod u+x files

上传文件不存在，也是由于目录权限导致 ./files/default无权限导致

建议代码中所有os.mkdir权限全部改为0775

今天又试了试，我甚至重新部署了一遍，去重功能也关了，早上 在files/0709/chaxiao8ml/870路径下上传了870张照片， 在files/0709/chaxiaoml/844路径下上传了844张照片， 在files/0709/test路径下上传了1张照片，调用statAPI统计fileCount一共1715张，一切都很正常， 但是下午一看，870张照片变成了824张，844张照片变成了808张，更离谱的是调用statAPI统计fileCount变成了1721张，不少反多？这些数据都是刚测出来，搞不懂为什么https://github.com/sjqzhang/go-fastdfs/issues/293

StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.HttpConnection+HttpConnectionResponseContent, Headers:\r\n{\r\n Access-Control-Allow-Headers: Authorization, Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, X-Requested-By, If-Modified-Since, X-File-Name, X-File-Type, Cache-Control, Origin\r\n Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE\r\n Access-Control-Allow-Origin: *\r\n Access-Control-Expose-Headers: Authorization\r\n Tus-Resumable: 1.0.0\r\n X-Content-Type-Options: nosniff\r\n Date: Mon, 24 May 2021 13:01:18 GMT\r\n Content-Length: 17\r\n Content-Type: text/plain; charset=utf-8\r\n}

1、add go mod support 2、reconstruct project 3、add github workflows 4、add Makefile 5、change tusd import path 6、change gopsutil import path 7、add .gitignore file 8、add cmd

// run on mac
$ ./fileserver_darwin                                                                                                                                                                                                                                                                                                                               tdbackend@MSMac
2021/05/17 18:02:10 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
Usage:
  fileserver [command]

Available Commands:
  doc         Run doc server
  help        Help about any command
  server      Run fastdfs server
  version     version

Flags:
  -h, --help   help for fileserver

Use "fileserver [command] --help" for more information about a command.

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@an5er) has found a potential issue, which I would be eager to share with you.

Could you add a SECURITY.md file with an e-mail address for me to send further details to? GitHub recommends a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

我是按demo上的宽高进行调试url地址的,调整后图挂了,

<img src="http://localhost:8080/group1/default/20220902/17/02/0/2.jpg?download=0&height=100&width=100">

1、上传编码后的中文文件名，本地文件存储为编码后的文件名 2、返回的url也是编码后的，但发现访问URL时，会解码URI，导致无法访问到 fileserver.go:285

if fullpath, err = url.PathUnescape(fullpath); err != nil {
		log.Error(err)
}