Caddy-git - Git Plugin for Caddy v2

caddy-git

Git Plugin for Caddy v2.

Inspired by this comment.

Please ask questions either here or via LinkedIn. I am happy to help you! @greenpau

Please see other plugins:

Table of Contents

Overview

The caddy-git allows updating a directory backed by a git repo.

Getting Started

For example, the following configuration sets up a definition for authp.github.io repo. The request to authp.myfiosgateway.com/update/authp.github.io trigger git pull of the authp.github.io repository.

{
  git {
    repo authp.github.io {
      base_dir /tmp
      url https://github.com/authp/authp.github.io.git
      branch gh-pages
      depth 1
    }
  }
}

authp.myfiosgateway.com {
  route /update/authp.github.io {
    git update repo authp.github.io
  }
  route {
    file_server {
      root /tmp/authp.github.io
    }
  }
}

The cloning of the repository happens on startup. Additionally, the cloning happens when /update/authp.github.io is being hit.

curl https://authp.myfiosgateway.com/update/authp.github.io
Owner
Paul Greenberg
Love tinkering with tech!
Paul Greenberg
Comments
  • Error after running githook

    Error after running githook

    Describe the issue

    When processing the githook, I am getting the following error.

    Jan 14 16:14:28 nycaddy caddy[192921]: {"level":"warn","ts":1642194868.222069,"logger":"http.handlers.git","msg":"failed updating repo","repo_name":"aspecthq.com","error":"object not found"}
    

    Configuration

    Paste full Caddyfile below:

    {
        git {
            repo aspecthq.com {
                base_dir /var/www
                url https://gitlab.com/joeworkman/aspecthq.com.git
                branch master
                depth 1
                # auth ssh /var/www/ssh/id_rsa.pub
                webhook name X-Gitlab-Token go-caddy-git-go
                # post pull exec {
                #     name Pager
                #     command /usr/local/bin/pager
                #     args "pulled authp.github.io repo"
                # }
            }
        }
    }
    aspecthq.com {
        redir https://www.aspecthq.com{uri}
    }
    www.aspecthq.com {
        log {
            output file /var/log/caddy/aspecthq.com/access.log {
                roll_size 3MiB
                roll_keep 5
                roll_keep_for 48h
            }
        }
        handle_errors {
            respond "{http.error.status_code} {http.error.status_text}"
            # rewrite * /{http.error.status_code}.html
            # file_server
        }
        route /update {
            git update repo aspecthq.com
        }
        encode gzip zstd
        php_fastcgi unix//run/php/php8.1-fpm.sock
        file_server
        root * /var/www/aspecthq.com
        push
    }
    

    Version Information

    Provide output of caddy list-modules -versions | grep git below:

    git v1.0.4
    http.handlers.git v1.0.4
    

    Expected behavior

    I would expect a pull request to be made

    Additional context

    https://github.com/greenpau/caddy-git/issues/1#issuecomment-1013469973

  • Post pull command error

    Post pull command error

    Describe the issue

    Post pull exec command error

    Configuration

    {
    	git {
    		repo github-vite-demo {
    			base_dir /tmp/ssh
    			url https://github.com/vfiee/github-vite-demo.git
    			branch main
    			auth key /root/.ssh/id_rsa
    			post pull exec {
    				name changeDir
    				command /bin/cd
    				args "/tmp/ssh/github-vite-demo"
    			}
    			post pull exec {
    				name install
    				command /usr/local/bin/npm
    				args "run install"
    			}
    			post pull exec {
    				name build
    				command /usr/local/bin/npm
    				args "run build"
    			}
    		}
    	}
    }
    
    :2022 {
    	root * /srv
    	encode gzip
    	file_server browse
    }
    
    :2023 {
    	route /update {
    		git update repo github-vite-demo
    	}
    	root * /tmp/ssh/github-vite-demo/dist
    	encode gzip
    	file_server browse
    }
    # https://caddyserver.com/docs/caddyfile
    
    # docker-compose
    version: "3"
    
    services:
      caddy:
        image: s6-caddy:latest
        container_name: caddy
        restart: always
        ports:
          - 2022:2022
          - 2019:2019
          - 2023:2023
        volumes:
          - ./var:/var
          - ./data:/data
          - /Users/vyron/.ssh:/root/.ssh
          - ./etc/caddy:/etc/caddy
          - ./etc/services:/etc/services.d
          - ../index.html:/srv/index.html
       
    # note: s6-caddy is my local docker image
    

    Version Information

    Provide output of caddy list-modules -versions | grep git below:

    / # caddy list-modules --versions | grep git
    git v1.0.7
    http.handlers.git v1.0.7
    

    Expected behavior

    Expect the command to run successfully

    Additional context

    # Dockerfile for build s6-caddy image
    
    FROM caddy:2.6.1-builder AS builder
    
    RUN xcaddy build \
        --with github.com/caddy-dns/cloudflare \
        --with github.com/greenpau/caddy-git
    
    FROM node:current-alpine3.16
    
    ENV TZ=Asia/Shanghai XDG_CONFIG_HOME=/config XDG_DATA_HOME=/data S6_VERSION=v2.2.0.3
    
    LABEL maintainer "vyronfiee <[email protected]>" \
        s6_version="${S6_VERSION}" \
        caddy_version="2.6.1"
    
    COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    
    RUN apk add --no-cache \
        ca-certificates \
        git \
        mailcap \
        openssh-client \
        tzdata \
        inotify-tools \
        curl && \ 
        curl -fSL https://github.com/just-containers/s6-overlay/releases/download/$S6_VERSION/s6-overlay-amd64.tar.gz | tar xz && \
        /usr/bin/caddy version && \
        /usr/bin/caddy list-modules && \
        echo 'Asia/Shanghai' >/etc/timezone 
    
    EXPOSE 80 443 2019
    
    VOLUME /srv
    
    WORKDIR /srv
    
    COPY Caddyfile /etc/caddy/Caddyfile
    
    COPY index.html /srv/index.html
    
    ENTRYPOINT ["/init"]
    
  • feature: Polling of repo for changes at a given interval

    feature: Polling of repo for changes at a given interval

    A clear and concise description of what you want the system to do.

    Polling to allow for private deployments to check repository for changes to trigger a pull.

    What are the Caddyfile directives that need to be added.

    Add Caddyfile directive:

    git {
      repo foo {
        interval <seconds>
      }
    }
    

    I love the work you've done on this, I was quite disappointed when I found that references online to http.git were for caddy v1 so this really fills a void.

    For servers that have open access it works great, but some internal work I do does not allow external traffic (for the webhook to hit) which is why the polling would be super beneficial.

  • breakfix: Git username prompt breaks building docker image with this plugin

    breakfix: Git username prompt breaks building docker image with this plugin

    Hello, this error is thrown while building the Dockerfile below:

    2022/09/14 16:22:13 [INFO] exec (timeout=0s): /usr/local/go/bin/go get -d -v github.com/greenpau/caddygit/module/git github.com/caddyserver/caddy/[email protected] 
    go: module github.com/greenpau/caddygit/module/git: git ls-remote -q origin in /go/pkg/mod/cache/vcs/49b5969af648321b8e1b610bca7e23162a006c17f126d617be4906822618e6bc: exit status 128:
    	fatal: could not read Username for 'https://github.com': terminal prompts disabled
    Confirm the import path was entered correctly.
    If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.
    2022/09/14 16:22:16 [FATAL] exit status 1
    

    Dockerfile

    FROM caddy:builder AS builder
    
    RUN xcaddy build \
        --with github.com/greenpau/caddygit/
    
    FROM caddy:latest
    
    COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    

    Configuration

    Paste full Caddyfile below:

    {
        email REDACTED
    
        git {
            repo REDACTED.com {
                base_dir /srv/REDACTED
                url https://github.com/REDACTED/REDACTED.git
                webhook Github X-Hub-Signature-256 {$WEBHOOK_SECRET}
                branch main
            }
        }
    }
    
    REDACTED.com, www.REDACTED.com {
        root * /srv/REDACTED
    
        route /update {
            git update repo REDACTED.com
        }
    
        route {
            file_server {
                hide .git*
            }
        }
    }
    
    REDACTED.REDACTED.com {
      encode gzip
    
      reverse_proxy /notifications/hub REDACTED:REDACTED
    
      reverse_proxy REDACTED:REDACTED {
           header_up X-Real-IP {remote_host}
      }
    }
    

    Version Information

    N/A, build issue

    Expected behavior

    Plugin should build

  • feature: add support for webhooks

    feature: add support for webhooks

    Per @joeworkman,

    webhook secret key is pretty simple. It's purely making sure that a specific header has a specific value. For Github the header is X-Hub-Signature-256. We simply need to compare that with the stored key. FYI. Gitlab's header for web hooks is X-Gitlab-Token

    Add Caddyfile directive:

    git {
      repo foo {
        webhook name <header> <secret>
      } 
    }
    

    When a request comes via API endpoint, inspect the header and authorize based on the presence of the webhook header and secret.

  • feature: Secure GitHub webhooks

    feature: Secure GitHub webhooks

    A clear and concise description of what you want the system to do.

    The webhooks feature cannot currently be used properly with Github. The reason for this is that the web hook header needs to be compared securely for each request. It's not as simple as just comparing the strings in the web hook header.

    Here are the docs for securing the web hooks for Github. https://docs.github.com/en/developers/webhooks-and-events/webhooks/securing-your-webhooks

  • feature: git matcher routes

    feature: git matcher routes

    A clear and concise description of what you want the system to do.

    It would be good to be able to define a matcher route instead of defining an explicit route.

    What are the Caddyfile directives that need to be added.

    Current:

    route /update {
        git update repo aspecthq.com
    }
    

    Requested:

    git /update update repo aspecthq.com
    
  • feature: add support for ssh key authentication

    feature: add support for ssh key authentication

    A clear and concise description of what you want the system to do.

    Add support for ssh key authentication.

    Github docs: https://docs.github.com/en/authentication/connecting-to-github-with-ssh

    What are the Caddyfile directives that need to be added.

    Add Caddyfile directive:

    git {
      repo foo {
        url [email protected]:greenpau/caddy-git.git
        auth key <path> [passphrase <secret>]
      }
    }
    
  • breakfix: empty git-upload-pack given

    breakfix: empty git-upload-pack given

    Receiving the following error when pulling an existing repo:

    run: loading initial config: loading new config: loading git app module: provision git: empty git-upload-pack given
    

    The error is being triggered invoking Pull() function:

    https://github.com/greenpau/caddy-git/blob/10e5d772a3708d65d8fb28a200b16bc77c35019d/pkg/service/repo.go#L80-L117

  • Run commands in background

    Run commands in background

    I run my static site builder as a post commands, and without running the command in the background it pretty much downs my site for a few seconds every time the command is executed.

  • breakfix: Interval not reset on config reload

    breakfix: Interval not reset on config reload

    Describe the issue

    @greenpau I've been using this and it works great, one little bug though that may be due to my own config.

    I have the --watch flag run with caddy to update on config changes, and was recently testing out a new server config. It looks like the timer from the prior config does not end when a new one is loaded.

    A quick test will allow you to see, set "update every 1" in the config and then edit it to "update every 60", config will reload but the old timer will still run. If you repeat this, even for an unrelated config change, multiple timers can be seen running.

    Let me know if you need any further details.

  • breakfix: Post Pull running on every iteration

    breakfix: Post Pull running on every iteration

    Describe the issue

    When using 'update every' to check a repository every x seconds, the post pull command is running even when the repository hasn't been changed.

    Expected behavior

    It would be great if post pull only runs when the repository was updated.

  • question: How well does this work with local repos?

    question: How well does this work with local repos?

    Hello!

    I'm currently putting together a personal server VM with Gitea and Caddy. I'm wondering if it might be possible to have my homepage and blog's static sources as a git repo be used by caddy with this plugin? E.g. /var/lib/gitea/repositories/tec/website.git. For non-static pages I'm thinking a dedicated "static/generated content" branch could be generated by CI. I'm well aware this would not be a standard approach, but for my situation (and maybe others?), it seems like this could be a convenient way of setting things up.

    Does this use case fit well with this plugin?

  • Repo not cloned if directory already exists

    Repo not cloned if directory already exists

    Describe the issue

    I noticed that if I already have the folder created (/var/www/aspecthq.com), then the repo would not get initialized. I have to run the initial git clone myself. Not a huge deal. I would expect it to work though.

    Configuration

    Paste full Caddyfile below:

    {
        git {
            repo aspecthq.com {
                base_dir /var/www
                url https://gitlab.com/joeworkman/aspecthq.com.git
                branch master
                depth 1
                # auth ssh /var/www/ssh/id_rsa.pub
                webhook name X-Gitlab-Token go-caddy-git-go
                # post pull exec {
                #     name Pager
                #     command /usr/local/bin/pager
                #     args "pulled authp.github.io repo"
                # }
            }
        }
    }
    aspecthq.com {
        redir https://www.aspecthq.com{uri}
    }
    www.aspecthq.com {
        log {
            output file /var/log/caddy/aspecthq.com/access.log {
                roll_size 3MiB
                roll_keep 5
                roll_keep_for 48h
            }
        }
        handle_errors {
            respond "{http.error.status_code} {http.error.status_text}"
            # rewrite * /{http.error.status_code}.html
            # file_server
        }
        route /update {
            git update repo aspecthq.com
        }
        encode gzip zstd
        php_fastcgi unix//run/php/php8.1-fpm.sock
        file_server
        root * /var/www/aspecthq.com
        push
    }
    

    Version Information

    Provide output of caddy list-modules -versions | grep git below:

    git v1.0.4
    http.handlers.git v1.0.4
    
Related tags
HTTP rate limiting module for Caddy 2

This module implements both internal and distributed HTTP rate limiting. Requests can be rejected after a specified rate limit is hit.

Jan 3, 2023
IONOS DNS module for caddy

This package contains a DNS provider module for Caddy. It is used to manage DNS records with the IONOS DNS API using libdns-ionos..

Nov 9, 2022
Service Management App for Caddy v2

caddy-systemd Service Management App for Caddy v2. Please ask questions either here or via LinkedIn. I am happy to help you! @greenpau Please see othe

Sep 1, 2022
Watch for interesting patterns in Caddy logs and send a Telegram notification.

Watch for interesting patterns in Caddy logs and send a Telegram notification.

Jul 21, 2022
Access ftp through caddy

Access ftp through caddy

Dec 14, 2022
Enable requests served by caddy for distributed tracing via The OpenTracing Project.

caddy-opentracing Enable requests served by caddy for distributed tracing via The OpenTracing Project. Dependencies The Go OpenTracing Library Jaeger,

Sep 30, 2022
Caddy log filter module with a log field filter to extract the user from a basic Authorization HTTP-Header

caddy-basic-auth-filter This packages contains a log field filter to extract the user from a basic Authorization HTTP-Header. Installation xcaddy buil

May 10, 2022
netcup DNS module for caddy: dns.providers.netcup

netcup DNS module for Caddy This package contains a DNS provider module for Caddy. It can be used to manage DNS records with the netcup DNS API using

Nov 9, 2022
A Caddy v2 extension to apply rate-limiting for HTTP requests

ratelimit A Caddy v2 extension to apply rate-limiting for HTTP requests. Installation $ xcaddy build --with github.com/owlwang/caddy-ratelimit Caddyfi

Jan 28, 2022
The plugin serves as a starting point for writing a Mattermost plugin

Plugin Starter Template This plugin serves as a starting point for writing a Mattermost plugin. Feel free to base your own plugin off this repository.

Dec 10, 2021
Feb 10, 2022
Cf-cli-find-app-plugin - CF CLI plugin to find applications containing a search string

Overview This cf cli plugin allows users to search for application names that co

Jan 3, 2022
Twitter-plugin - Falco Plugin for Twitter Stream

Twitter Plugin This repository contains the twittter plugin for Falco, which fol

Mar 17, 2022
Examples using the stomp package from git://github.com/gmallard/stompngo.git

stompngo_examples - A collection of examples for package stompngo Features Full demonstration of support for STOMP protocols: Protocol Level 1.0 Proto

Jan 22, 2021
one simple git ssh server (just for learning git over ssh )

wriet one simple git ssh server use golang write one simple git ssh server how to running starting service docker-compose up -d add authorized_keys i

Mar 5, 2022
A Go library for connecting to HandlerSocket (github.com/ahiguti/HandlerSocket-Plugin-for-MySQL)

handlersocket-go Go library for connecting to HandlerSocket Mysql plugin. See github.com/ahiguti/HandlerSocket-Plugin-for-MySQL/ Installation $ go get

Jan 19, 2021
Yet another SIP003 plugin for shadowsocks, based on Xray-core

Yet another SIP003 plugin for shadowsocks, based on Xray-core Build go build Usage See command line args for advanced usages.

Jan 8, 2023
DipDup plugin for selective metadata indexing

DipDup metadata indexer DipDup service for indexing contract and token metadata. Based on TzKT indexer. For start synchronization of DipDup state TzKT

Nov 25, 2022
Tool for monitoring network devices (mainly using SNMP) - monitoring check plugin
Tool for monitoring network devices (mainly using SNMP) - monitoring check plugin

Thola Description A tool for monitoring network devices written in Go. It features a check mode which complies with the monitoring plugins development

Dec 29, 2022