HTTP based Tree-shaped Peer2Peer blob transfer proxy, distributing images or blob data.

DadiP2P

DadiP2P is an accelerator that uses P2P protocol to speed up HTTP file download, usually use for docker image layer download.

The key features are:

  • High Performance

    We use P2P protocol and chunked-file transfer. Use P2P protocol can reduce the pressure on the origin, and use chunked-file can allow client parallel download from the P2P network. These initiatives can dramatically increase transfer speed. We also use LRU cache to store chunked-file to speed up file reading and reduce file storage space.

  • High Reliability

    When any node unserviceable, we will automatic redirect to other node. And if current node is shutdown, we will automatic restart this node and try to rebuild LRU cache.

Getting Started

Build From Source

Requirements
  • Golang >= 1.16.x
make

The DadiP2P binary are generated in bin.

If you are in China mainland and download gomod failed, you can use goproxy to speed up download.

export GOPROXY=https://mirrors.aliyun.com/goproxy/

Configure

The default configure file is dadip2p.yaml, you can direct edit it.

Run

./p2p -c dadip2p.yaml

Overview

DadiP2P consists of a proxy server and a P2P server.

Proxy server will hijack users' requests. if match rule, proxy will redirect the request to P2P server.

Proxy server support forward proxy and reverse proxy, see PROXY_MODE.

P2P server have two types of nodes: root node and agent node, all nodes are available for external service.

  • Only root node will download from origin.
  • Agent node will download from root node or redirect to another agent node.

p2p-framework.png

Licenses

Accelerated Container Image is released under the Apache License, Version 2.0.

Similar Resources

An experimental Tor-Proxy serivce written in Go using Go-proxy and Go-libtor.

tor-proxy An experimental standalone tor-proxy service built with Go, using go-proxy, go-libtor and bine. This is a simple replacement to Tor's origin

Nov 9, 2022

mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together

mt-multiserver-proxy mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together. It is the successor to multiserv

Nov 17, 2022

IP2Proxy Go package allows users to query an IP address to determine if it was being used as open proxy, web proxy, VPN anonymizer and TOR exits.

IP2Proxy Go Package This package allows user to query an IP address if it was being used as VPN anonymizer, open proxies, web proxies, Tor exits, data

Sep 15, 2022

Tcp-proxy - A dead simple reverse proxy server.

tcp-proxy A proxy that forwords from a host to another. Building go build -ldflags="-X 'main.Version=$(git describe --tags $(git rev-list --tags --max

Jan 2, 2022

Proxy - Minimalistic TCP relay proxy.

Proxy Minimalistic TCP relay proxy. Installation ensure you have go = 1.17 installed clone the repo cd proxy go install main.go Examples Listen on po

May 22, 2022

Battlesnake-logging-proxy - A little proxy between the internet and your battlesnake

battlesnake-logging-proxy a little proxy between the internet and your battlesna

Feb 11, 2022

Go HTTP tunnel is a reverse tunnel based on HTTP/2.

Go HTTP tunnel is a reverse tunnel based on HTTP/2. It enables you to share your localhost when you don't have a public IP.

Dec 28, 2022

V3IO Frames ("Frames") is a Golang based remote data frames access (over gRPC or HTTP stream)

V3IO Frames ("Frames") is a multi-model open-source data-access library that provides a unified high-performance DataFrame API for working with different types of data sources (backends). The library was developed by Iguazio to simplify working with data in the Iguazio Data Science Platform ("the platform"), but it can be extended to support additional backend types.

Oct 1, 2022

Goket (Golang Keyboard Event Tree) is a proof-of-concept code for using keyboard events trees for performing operations.

Goket Goket (Golang Keyboard Event Tree) is a proof-of-concept code for using keyboard events trees for performing operations. Its main goal is to all

Jan 3, 2023
Comments
  • Does the code work? Is there deployment documentation?

    Does the code work? Is there deployment documentation?

    hi, does this code work fine? When the container engine pulls the image, the uri is generally like this: /v2/ImageName/blobs/sha256:xxxxx , but the code is like this:https://github.com/data-accelerator/dadi-p2proxy/blob/cc3f17655cf93fbb35d892250f8074177f5b073b/pkg/p2p/server/proxy_server.go#L41

Related tags
📦 Command line peer-to-peer data transfer tool based on libp2p.

pcp - Peer Copy Command line peer-to-peer data transfer tool based on libp2p. Table of Contents Motivation Project Status How does it work? Usage Inst

Jan 5, 2023
A MCBE Proxy supporting fast transfer and much more!
A MCBE Proxy supporting fast transfer and much more!

Downloads Pipelines Here you can find all the build please select the latest and click Artifacts

Oct 17, 2022
Transfer 10Gbps http traffic over 1Gbps networks :)

httpteleport Teleports 10Gbps http traffic over 1Gbps networks. Built on top of fastrpc. Use cases httpteleport may significantly reduce inter-server

Nov 30, 2022
A simple tool to convert socket5 proxy protocol to http proxy protocol

Socket5 to HTTP 这是一个超简单的 Socket5 代理转换成 HTTP 代理的小工具。 如何安装? Golang 用户 # Required Go 1.17+ go install github.com/mritd/s2h@master Docker 用户 docker pull m

Jan 2, 2023
Client-Server tcp-based file transfer application in GoLang

Клиент-серверный файловый сервис на базе протокола TCP Клиент client.go шифрует свои файлы алгоритмом AES с режимом CBC и помещает их на сервер server

Apr 1, 2022
Go-http-sleep: Delayed response http server, useful for testing various timeout issue for application running behind proxy

delayed response http server, useful for testing various timeout issue for application running behind proxy

Jan 22, 2022
A CLI-based HTTP intercept and replay proxy
A CLI-based HTTP intercept and replay proxy

GLORP Glorp is an HTTP intercept proxy, allowing the inspection and replaying of HTTP requests. The layout and flow was designed to function similar t

Dec 14, 2022
Transfer files between machines in the same network

GoTrans Transfer files from computer A to computer B. The program have a sender and a receiver, if you choose a sender you'll need to provide the ip a

Nov 7, 2021
Bridge REMOV will allow you to safely transfer NFT from RMRK to MOVR and backwards

remov Inspiration Our aim is to expand the capabilities of blockchain and make a secure way for transferring NFT between RMRK and MOVR blockchain. The

Dec 5, 2021