Simple edge server / reverse proxy

reproxy build Coverage Status Go Report Card Docker Automated build

Reproxy is simple edge HTTP(s) sever / reverse proxy supporting various providers (docker, static, file). One or more providers supply information about requested server, requested url, destination url and health check url. Distributed as a single binary or as a docker container.

Server can be set as FQDN, i.e. s.example.com or * (catch all). Requested url can be regex, for example ^/api/(.*) and destination url may have regex matched groups in, i.e. http://d.example.com:8080/$1. For the example above http://s.example.com/api/something?foo=bar will be proxied to http://d.example.com:8080/something?foo=bar.

For convenience, requests with the trailing / and without regex groups expanded to /(.*), and destinations in those cases expanded to /$1. I.e. /api/ -> http://127.0.0.1/service will be translated to ^/api/(.*) -> http://127.0.0.1/service/$1

Both HTTP and HTTPS supported. For HTTPS, static certificate can be used as well as automated ACME (Let's Encrypt) certificates. Optional assets server can be used to serve static files.

Starting reproxy requires at least one provider defined. The rest of parameters are strictly optional and have sane default.

example with a static provider:

reproxy --static.enabled --static.rule="example.com/api/(.*),https://api.example.com/$1"

Install

  • for a binary distribution pick the proper file in the release section
  • docker container available via docker hub (umputun/reproxy) as well as via github container registry (ghcr.io/umputun/reproxy). Latest stable version has :vX.Y.Z tag (with :latest alias) and the current master has :master tag.

Providers

User can sets multiple providers at the same time.

See examples of various providers in examples

Static

This is the simplest provider defining all mapping rules directly in the command line (or environment). Multiple rules supported. Each rule is 3 or 4 comma-separated elements server,sourceurl,destination,[ping-url]. For example:

  • *,^/api/(.*),https://api.example.com/$1 - proxy all request to any host/server with /api prefix to https://api.example.com
  • example.com,/foo/bar,https://api.example.com/zzz - proxy all requests to example.com and with /foo/bar url to https://api.example.com/zzz

The last (4th) element defines an optional ping url used for health reporting. I.e.*,^/api/(.*),https://api.example.com/$1,https://api.example.com/ping. See Health check section for more details.

File

reproxy --file.enabled --file.name=config.yml

example of config.yml:

default: # the same as * (catch-all) server
  - {route: "^/api/svc1/(.*)", dest: "http://127.0.0.1:8080/blah1/$1"}
  - {route: "/api/svc3/xyz", dest: "http://127.0.0.3:8080/blah3/xyz", "ping": "http://127.0.0.3:8080/ping"}
srv.example.com:
  - {route: "^/api/svc2/(.*)", dest: "http://127.0.0.2:8080/blah2/$1/abc"}

This is a dynamic provider and file change will be applied automatically.

Docker

Docker provider works with no extra configuration and by default redirects all requests like https://server/api/<container_name>/(.*) to the internal IP of the given container and the exposed port. Only active (running) containers will be detected.

This default can be changed with labels:

  • reproxy.server - server (hostname) to match
  • reproxy.route - source route (location)
  • reproxy.dest - destination path. Note: this is not full url, but just the path which will be appended to container's ip:port
  • reproxy.ping - ping path for the destination container.

By default all containers with exposed port will be considered as routing destinations. There are 2 ways to restrict it:

  • Exclude some containers explicitly with --docker.exclude, i.e. --docker.exclude=c1 --docker.exclude=c2 ...
  • Allow only a particular docker network with --docker.network

This is a dynamic provider and any change in container's status will be applied automatically.

SSL support

SSL mode (by default none) can be set to auto (ACME/LE certificates), static (existing certificate) or none. If auto turned on SSL certificate will be issued automatically for all discovered server names. User can override it by setting --ssl.fqdn value(s)

Logging

By default no request log generated. This can be turned on by setting --logger.enabled. The log (auto-rotated) has Apache Combined Log Format

Assets Server

User may turn assets server on (off by default) to serve static files. As long as --assets.location set it will treat every non-proxied request under assets.root as a request for static files.

Assets server can be used without any proxy providers. In this mode reproxy acts as a simple web server for a static context.

More options

  • --gzip enables gizp compression for responses.
  • --max=N allows to set the maximum size of request (default 64k)
  • --header sets extra header(s) added to each proxied request

Ping and health checks

reproxy provides 2 endpoints for this purpose:

  • /ping responds with pong and indicates what reproxy up and running
  • /health returns 200 OK status if all destination servers responded to their ping request with 200 or 417 Expectation Failed if any of servers responded with non-200 code. It also returns json body with details about passed/failed services.

All Application Options

  -l, --listen=                     listen on host:port (default: 127.0.0.1:8080) [$LISTEN]
  -t, --timeout=                    proxy timeout (default: 5s) [$TIMEOUT]
  -m, --max=                        max response size (default: 64000) [$MAX_SIZE]
  -g, --gzip                        enable gz compression [$GZIP]
  -x, --header=                     proxy headers [$HEADER]
      --no-signature                disable reproxy signature headers [$NO_SIGNATURE]
      --dbg                         debug mode [$DEBUG]

ssl:
      --ssl.type=[none|static|auto] ssl (auto) support (default: none) [$SSL_TYPE]
      --ssl.cert=                   path to cert.pem file [$SSL_CERT]
      --ssl.key=                    path to key.pem file [$SSL_KEY]
      --ssl.acme-location=          dir where certificates will be stored by autocert manager (default: ./var/acme) [$SSL_ACME_LOCATION]
      --ssl.acme-email=             admin email for certificate notifications [$SSL_ACME_EMAIL]
      --ssl.http-port=              http port for redirect to https and acme challenge test (default: 80) [$SSL_HTTP_PORT]
      --ssl.fqdn=                   FQDN(s) for ACME certificates [$SSL_ACME_FQDN]

assets:
  -a, --assets.location=            assets location [$ASSETS_LOCATION]
      --assets.root=                assets web root (default: /) [$ASSETS_ROOT]

logger:
      --logger.enabled              enable access and error rotated logs [$LOGGER_ENABLED]
      --logger.file=                location of access log (default: access.log) [$LOGGER_FILE]
      --logger.max-size=            maximum size in megabytes before it gets rotated (default: 100) [$LOGGER_MAX_SIZE]
      --logger.max-backups=         maximum number of old log files to retain (default: 10) [$LOGGER_MAX_BACKUPS]

docker:
      --docker.enabled              enable docker provider [$DOCKER_ENABLED]
      --docker.host=                docker host (default: unix:///var/run/docker.sock) [$DOCKER_HOST]
      --docker.network=             docker network (default: bridge) [$DOCKER_NETWORK]
      --docker.exclude=             excluded containers [$DOCKER_EXCLUDE]

file:
      --file.enabled                enable file provider [$FILE_ENABLED]
      --file.name=                  file name (default: reproxy.yml) [$FILE_NAME]
      --file.interval=              file check interval (default: 3s) [$FILE_INTERVAL]
      --file.delay=                 file event delay (default: 500ms) [$FILE_DELAY]

static:
      --static.enabled              enable static provider [$STATIC_ENABLED]
      --static.rule=                routing rules [$STATIC_RULES]

Help Options:
  -h, --help                        Show this help message
  

Status

The project is under active development and may have breaking changes till v1 released.

Comments
  • Performance with file uploads

    Performance with file uploads

    a simple 12Mb file takes 1.6 sec to upload with reproxy

    With reproxy:

    time curl -X POST "http://localhost/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "[email protected]"
    
    0.01s user 0.01s system 1% cpu 1.614 total
    

    without :

    time curl -X POST "http://localhost:8000/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "[email protected]"
    
    0.00s user 0.01s system 29% cpu 0.062 total
    

    conf:

      reproxy:
        image: umputun/reproxy:master
        container_name: reproxy
        hostname: reproxy
        ports:
          - "80:8080"
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock:ro
          - ./files/static:/static
        environment:
          - LISTEN=0.0.0.0:8080
          - DOCKER_ENABLED=true
          - DEBUG=true
          - MAX_SIZE=20000000
    
  • [question] is there a way to serve static files from multiple locatons ?

    [question] is there a way to serve static files from multiple locatons ?

    Is it possible to make some mapping like this:

    - site1.com/api/    ---> docker.container1
    - site1.com/static/ ---> static path /var/www/staticfolder1
    - site2.com/api/    ---> docker.container2
    - site2.com/static/ ---> /var/www/anotherfolder
    
  • File event tests may occasionally fail

    File event tests may occasionally fail

    It looks like certain tests that rely on short timings may occasionally fail. This is coming from downstream NixOS CI, and in this case on macOS: https://hydra.nixos.org/build/143298376/log

    But I was unable to reproduce this locally on macOS, either in a normal build or in a build using Nix. I'm guessing it's a matter of tight tolerances and busy CI machine, maybe?

    Do you think it's safe to ignore these failures?

  • Unable to specify header with ',' in value

    Unable to specify header with ',' in value

    I want to add

    Access-Control-Allow-Headers:DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type

    but in response I got

    Access-Control-Allow-Headers:DNT

    if I specify

    Access-Control-Allow-Headers:"DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type"

    I got Access-Control-Allow-Headers:"DNT

    for Access-Control-Allow-Headers:'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'

    I got Access-Control-Allow-Headers:'DNT

  • Add a simple throttler/limiter

    Add a simple throttler/limiter

    This can be useful, and won't add much complexity. The throttler should be able to limit the total number of concurrent requests per reproxy server as well as per proxy destination. Not sure if limiting per destination server (virtual host) needed.

    It also should be able to limit req/sec per remote client for the same areas (total and destination).

  • Support multiple route pairs in docker labels

    Support multiple route pairs in docker labels

    re: https://github.com/umputun/reproxy/discussions/76

    In some cases, one container may expose multiple endpoints, for example, public API and some admin API. They can be on different ports and the goal is to allow such functionality to be defined with docker provider (labels?)

    I can think of two ways to implement it:

    • add numeric suffix to reproxy.*.N, i.e. reproxy.server.1=example.com, reproxy.dest.1=blah and so on. This may increase the number of labels and could be ugly in extreme cases
    • similar to the first approach but combine the rule in this style of static provider, i.e. reproxy.1=m.example.com,/api/(.*)./someplace/$1,/ping. This is harder to read but will minimize the number of labels

    both approaches can be combined too, and the user can choose which one he likes better. The current labels should be supported too as we don't want to break compatibility for no good reason

    if anyone has better ideas pls share

  • Replace fsouza/go-dockerclient with direct docker api calls

    Replace fsouza/go-dockerclient with direct docker api calls

    Just an idea - trying to minimize the number of third-party dependencies. The go-dockerclient seems to be a heavy one and brings a lot of transitive dependencies.

    All we do with docker is just event listener and container's list. It would be nice to replace go-dockerclient with internal calls to docker api

  • some services in docker examples do not work on ARM platform

    some services in docker examples do not work on ARM platform

    When used on ARMv7 (odroid c1) all services works with exception of svc1 and svc2 that are based on amd64's hashicorp/http-echo. Probably it will be better to use a multi-platform http-echo image for svc1/svc2.

    Example of the problem log:

    # docker logs svc1
    standard_init_linux.go:219: exec user process caused: exec format error
    

    I researched for some other images and found teapow/http-echo which worked for me.

    Example of docker-compose.yml insert:

      echo1:
        image: teapow/http-echo
        container_name: echo1
        hostname: echo1
        ports:
          - "9090:9090"
        environment:
          - SERVER_PORT=9090
          - ECHO_MESSAGE="teapow pong"
          - SERVER_DEBUG=true
    

    test:

    # curl http://localhost/api/echo1/
    {
      "message": "\"teapow pong\""
    }
    
  • Simplify usage (based on the real-life setup)

    Simplify usage (based on the real-life setup)

    I have tried to migrate one of my non-trivial configurations hosting multiple docker services with nginx-le proxy in front of this and with several static sites to reproxy. The good news - I was able to do most of it and the final configuration seems to be easier. No need for multiple configuration files (i.e. docker-compose + nginx configs) and generally the final compose readable and make sense.

    However, I've run into some unexpected complexity and most of it was self-induced and hopefully can be addressed with relatively easy changes:

    1. Mix of static rules and docker provider doesn't seem to work together. Not sure why, need to investigate
    2. All of my destination containers needed reproxy.route: /(.*) and reproxy.dest: /@1 labels and this pair seems to be a good candidate for a new default
    3. In order to allow the current label-less docker discovery I'm going to add a new bool parameter turning it on or off. Not sure which makes more sense as a default.
    4. Some containers needed to handle multiple servers, i.e. blah.com and www.blah.com, but currently, only a single server name supported. Should be doable to support multiple servers
    5. Attempt to serve fully static content with separate reproxy containers (see #12) worked, but looks a little bit odd. I'm not sure if I like it or maybe prefer some other approach
    6. I had to set LISTEN=0.0.0.0:port to each reproxy container because the default is localhost:port. From safety/security POV it makes sense, but practically every reproxy container will have to have LISTEN define explicitly. Maybe this is the good thing, not sure
    7. Additional reproxy servers (for static content) needed reproxy.enabled: y because the container itself sets it to false (see #16). This is logical (technically speaking) but unexpected and confusing in practice.
  • Support regex in host / server

    Support regex in host / server

    Main consideration is backward compatibility. example.com should be treated as an exact match, where possible. So current order is: exact host, regex host, * or ""

  • Cache and Expires by Media Type

    Cache and Expires by Media Type

    ASSETS_CACHE now sets only a Cache-control header for all static assets — images, CSS, scripts, and HTML. It would be useful at least to have an option to exclude HTML from the cache.

    More obvious way - to add Expires header and an option to set it by MediaType.

    I see index HTML pages served from the browser cache in the current configuration even after Hugo's rebuild - because max-age from Cache-control yet to be expired.

  • Fix redirects

    Fix redirects

    Before this change redirects didn't work because method Service.extendMapper didn't copy the value of URLMapper.RedirectType to the extended result.

    To fix this we return the original URLMapper instead of creating a new one (it can also help to avoid similar bugs in the future). We can reuse URLMapper because it is passed by value.

  • trivy action and configuration

    trivy action and configuration

    Hi @umputun thanks for your work on reproxy.

    Please consider this GitHub workflow for using trivy to identify security vulnerabilities in reproxy.

    The action will run on commits to master and will identify any security issues identified by trivy against the docker image, it will use the config defined in the ./trivy.yaml for what it should do on the workflow execution. Currently, it is configured to identify Critical and High issues in the OS and Libraries.

    You can set the exit code to 0 if you don't want the workflow to fail if it identifies vuln's.

    Thanks, Damian.

  • page display error in readme.md

    page display error in readme.md

    There's a display error in your readme.md file: image

    The first two examples (static provider; automatic docker discovery) are truncated. May want to correct them. :-)

    Keep up the good work!

  • initial support for builtin plugins

    initial support for builtin plugins

    1. Clone plugin repo to folder plugins with folder name as plugin package name git clone https://github.com/negasus/reproxy-brotli-plugin.git plugins/brotli

    2. Remove go module files rm plugins/brotli/go.*

    3. Generate app/proxy/plugins.go go run ./cmd/plugins

    4. Vendoring go mod tidy go mod vendor

    5. Done Run reproxy for test

  • Feature request: per route timeout and throttle

    Feature request: per route timeout and throttle

    It would be great if I can set (in config file) timeout and throttle for specific routes. For example I use exponential timeouts for login form to protect from brute force and I'd like to increase timeouts for this endpoint.

  • Listen to IPv6 address

    Listen to IPv6 address

    Hey!

    Tried to set reproxy to listen on [::]:8080 (and :::8080 too) address but it doesn't want to start with some errors logged:

    reproxy  | 2022/03/31 21:17:03.924 [INFO]  activate http proxy server on "[::]:8080"
    reproxy  | 2022/03/31 21:17:03.925 [ERROR] proxy server failed, listen tcp: address "[::]:8080": too many colons in address
    reproxy  | 2022/03/31 21:17:03.925 [ERROR] proxy server failed, listen tcp: address "[::]:8080": too many colons in address
    

    Is IPv6 supported in reproxy? Could we implement if it isn't? Thanks!

Tcp-proxy - A dead simple reverse proxy server.

tcp-proxy A proxy that forwords from a host to another. Building go build -ldflags="-X 'main.Version=$(git describe --tags $(git rev-list --tags --max

Jan 2, 2022
mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together

mt-multiserver-proxy mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together. It is the successor to multiserv

Nov 17, 2022
4chain is a simple、fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet.

4Chain What is 4chain? 4chain is a simple、fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. Using the ssh

Nov 1, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

中文 MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

Dec 30, 2022
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

frp README | 中文文档 What is frp? frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. As of now, it s

Jan 5, 2023
Reverse proxy server to filter traffic based on JA3 fingerprint/hash

JA3RP (JA3 Reverse Proxy) Ja3RP is a basic reverse proxy server that filters traffic based on JA3 fingerprints. It can also operate as a regular HTTP

Sep 17, 2022
A standalone Web Server developed with the standard http library, suport reverse proxy & flexible configuration
A standalone Web Server developed with the standard http library, suport reverse proxy & flexible configuration

paddy 简介 paddy是一款单进程的独立运行的web server,基于golang的标准库net/http实现。 paddy提供以下功能: 直接配置http响应 目录文件服务器 proxy_pass代理 http反向代理 支持请求和响应插件 部署 编译 $ go build ./main/p

Oct 18, 2022
A paywall bypassing reverse proxy and DNS server written in go 🔨💵🧱

FreeNews ?? ?? ?? A paywall bypassing reverse proxy and DNS server written in go. This project is still hard work in progress. Expect stuff to just no

Dec 7, 2022
Dead simple reverse proxy for all your containerized needss
Dead simple reverse proxy for all your containerized needss

Whats this ? Pawxi is yet another reverse proxy designed with simplicity in mind. Born out of a certain users frustration at the complexity of setting

Oct 17, 2022
Simple Reverse Proxy Load Balancer

lb - a reverse proxy load-balancing server, It implements the Weighted Round Robin Balancing algorithm

Mar 23, 2022
Simple SNI based reverse proxy

SNIProxy SNIProxy is a very simple reverse proxy server that uses TLS SNI to route to hosts (in HTTP mode it just uses the Host header). It is designe

Oct 16, 2022
Simple Minecraft Bedrock reverse proxy

BedProx [WIP] Simple Minecraft Bedrock reverse proxy Features Reverse Proxy HAProxy Protocol Support (NOT TESTED) Webhooks REST API How to use/deploy

Sep 7, 2022
A simple small reverse proxy for golang

ssrp simple small reverse proxy bash <(curl -Ls https://raw.githubusercontent.com/go-bai/ssrp/master/install.sh) cn install bash <(curl -Ls https://gh

Apr 5, 2022
Gouter: Simple development reverse proxy

Gouter: Simple development reverse proxy

Jan 15, 2022
Reverse cwmp proxy
Reverse cwmp proxy

cwmp-proxy Integration of the proxy will provide you the ability to place CPEs and ACS servers in different networks. What about if the devices are pl

Nov 15, 2022
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator

Trickster is an HTTP reverse proxy/cache for http applications and a dashboard query accelerator for time series databases. Learn more below, and chec

Jan 2, 2023
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies

Weaver - A modern HTTP Proxy with Advanced features Description Features Installation Architecture Configuration Contributing License Description Weav

Dec 24, 2022
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies

Weaver - A modern HTTP Proxy with Advanced features Description Features Installation Architecture Configuration Contributing License Description Weav

Jan 1, 2023
gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era.
gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era.

gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era. Current status: Maintenance mode, accepting PRs. Currently in

Dec 25, 2022